Mathematical questions concerning floating point numbers, a finite approximation of the real numbers used in computing.

learn more… | top users | synonyms

0
votes
1answer
62 views

Gosper Formula for inv $\pi$, properties.

I need to understand very good how the properties of this formula $\frac{4}{\pi} = \frac{5}{4} + \sum_{N \geq 1} \left[ 2^{-12N + 1} \times(42N + 5)\times {\binom {2N-1} {N}}^3 \right] $ Taken from ...
5
votes
1answer
55 views

What are examples of cases where floating-point $aaaa\ne(aa)(aa)$?

As explained in answers to this question on SO, due to non-associativity of floating-point arithmetic repeated multiplication like $aaaa$ can't be optimized to $(aa)(aa)$. Of course, aside from just ...
1
vote
1answer
32 views

error bound in function approximation algorithm

Suppose we have the set of floating point number with "m" bit mantissa and "e" bits for exponent. Suppose more over we want to approximate a function "f". From the theory we know that usually a ...
0
votes
0answers
5 views

Prove that a condition for a product fitting in a range holds

Given the sets $$L = \{x \in \mathbb Z : -2^{63} \le x < 2^{63} \}$$ $$F = \{m \cdot 2^e \in \mathbb R : m \in \mathbb Z, -2^{24} + 1 \le m \le 2^{24} - 1, e \in \mathbb Z\}$$ and ...
2
votes
1answer
42 views

How to understand or derive the formula $Mantissa$ $bits$ $/$ ${log_2}$ $10$ = $Decimal$ $digits$ $of$ $precision$?

I asked a question a couple days ago about floating point precision on stackoverflow called, "Is floating point precision mutable or invariant?" and I received the following response. The formula ...
0
votes
2answers
53 views

Curve Fitting to Represent Any Data

I'm a programmer seeking to take a bunch of data and represent it as a curve. Specifically, I want to take several hundred/thousand (floating) points and represent those points to a specified level of ...
2
votes
1answer
66 views

IEEE-754 Format Conversion

Represent 11.0011 x 2^10 using the IEEE-754 standard for 32-bit floating point representation. 0-Sign Bit 10001010-Exponent 1001100000000000000000-Mantissa Is this answer is correct ? I am bit ...
11
votes
4answers
735 views

why is $2.2250738585072014\text{e}{-308}$ not a number? [closed]

In programming the min value of a float is: $$2.2250738585072014\text{e}{-308}$$ but when I type this into a calculator, it says Not a Number. what I am wondering ...
2
votes
0answers
35 views

Can trigonometric functions for double precision be implemented in terms of those for single precision?

In some program environments like GLSL there is support for single and double precision numbers for arithmetic and square roots computation, but only single precision trigonometric functions are ...
2
votes
1answer
74 views

Name of function $(1+x)^n-1$

Is there any name for this formula $$(1+x)^n-1$$ When working with floating point numbers this can be calculated with much better precision for very small $|x|<1$ values using Taylor series ...
0
votes
2answers
36 views

Simplifying a decimal number under modular arithmetic – $9.9 \pmod{13}$

Can you please help me simplify the relation $9.9 \pmod{13}$? It may seem like a stupid question (!) but your answers will help me very much. Thank you.
0
votes
1answer
16 views

approximation using floating point arithmetic

Let $x=2.14366$ and $y=2.14363$ and $d=x-y.$ If $d*$ is the value of d computed using $5-$digit decimal floating point arithmetic, find the relative error. For this question I know how to calculate ...
0
votes
1answer
54 views

Floating point number,Mantissa,Exponent

In this computer, numbers are stored in 12-bits. We will also assume that for a floating point (real) number, 6 bits of these bits are reserved for the mantissa (or significand) with 2^(k-1)-1 as the ...
2
votes
1answer
36 views

Audio frequency increment yielding wrong results

I am writing something in the ChucK programming language, which is designed specifically for audio time functions (in this case, hertz). I'm having a really difficult time with a mathematical ...
2
votes
3answers
60 views

Why do I get a big relative error for my function? (Numerical Analisys - floating point)

When evaluating on the computer the following function: $$f(x)=\frac{x^2}{(\cos(\sin(x)))^2-1}$$ there is a big relative error for values $x\approx0$ (values very close to zero). I used the Taylor ...
-2
votes
2answers
294 views

Convert decimal to Binary Floating Point - 8 Bit [closed]

I am trying to convert +3.5 to binary floating point, but im struggling to find the exponent. (8 Bit) Where 1st bit is the Sign, 3 bits for Exponent and 4 bits for Mantissa. Hope somebody can explain ...
0
votes
1answer
19 views

Precision and Accuracy

How would I go about calculating the precision and accuracy of a given number? For example 0.05 has an accuracy of 2 and a precision of 3. 1 has an accuracy of 0 and a precision of 1. Is ...
1
vote
2answers
71 views

How to calculate the inverse of sum of a Kronecker product and a diagonal matrix

I want to calculate the inverse of a matrix of the form $S = (A\otimes B+C)$, where $A$ and $B$ are symetric and invertible, $C$ is a diagonal matrix with positive elements. Basically if the ...
0
votes
0answers
52 views

Is is possible to define a sign convention for eigenvectors calculated with a small uncertainty?

I'm working with a numerical method that involves the diagonalization of a real, symmetric $n \times n$ matrix $H$. Now obviously the sign of the (normalized) eigenvectors $\phi_i$ is not well ...
0
votes
1answer
17 views

Storing a real matrix - fl notation

I have two matrix $A$ and $B$ and $fl(A), fl(B)$ denote the stored version them, respectively. Let $fl(AB)$ be the stored version of the product of $A$ and $B$. Is it true that $fl(AB) = fl(A) * ...
1
vote
0answers
27 views

What is the short name for a set of all floating-point numbers?

We can name the set of all real numbers $\mathbb{R}$, and the set of all integers $\mathbb{Z}$. Is there a commonly accepted short name for the set of all IEEE 754 floating point numbers? I understand ...
0
votes
1answer
363 views

Convert hexadecimal numbers to floating-point format using single-precision IEEE 754 format

I need help with converting hexadecimal numbers to floating-point format using single-precision IEEE 754 format. Hex numbers such as 312A. how do I go about converting it? I know how to convert from ...
0
votes
1answer
360 views

How to convert from floating point binary to decimal in half precision(16 bits)?

I'm trying to convert a 16 bit precision binary number to decimal format however I am completely failing to do so. The binary I'm trying to convert is $0101011101010000$ My current method is: ...
6
votes
1answer
149 views

Explain why catastrophic cancellation happens

After my own research, the following picture emerges as the most frequently used example of catastrophic cancellation (It is indeed used in my class). Could anyone explain why the plot takes that ...
0
votes
0answers
36 views

Binary to Decimal Floating Point Number

I seem to be having difficulties trying to figure this out: I have a Binary 0101011101010000 and I'm trying to compute the decimal floating point number, in the IEEE-754 format Can somebody help? ...
0
votes
1answer
46 views

How can I add the following 32-bit IEEE floating-point numbers?

How can I add the following two 32-bit IEEE floating-point numbers in binary? FEDCBA98(base 16) + 89ABCDEF(base 16) = a 33-bit binary number. How can this be possible?
0
votes
1answer
23 views

How would you convert the following 32-bit IEEE floating-point to decimal form?

I have got -1.101 1100 1011 1010 1001 1000 * 2^(9) How can I convert this to decimal form?
1
vote
1answer
47 views

Gaps between successive floating point numbers

(all numbers discussed are in decimal) lets say we have a floating point data type that is like : m * 10 ^ e ...
0
votes
2answers
33 views

Floating Point Calculation

Take the polynomial $x^2+(-4*10^3)x+2)$. On the floating point system, b=10, m=4, e=4, if I wanted to find the roots using the quadratic formula what would be the values of the roots? I got 3.999 as ...
0
votes
1answer
23 views

Selecting denominator for relative error margins

When looking at this page: http://floating-point-gui.de/errors/comparison/ there are values a, and b that are being compared ...
0
votes
0answers
16 views

Number Systems and Floating Point Arethmetic

I am confused about the term of finite precision number, is that the same with the floating point number? Moreover, I have a number system: ...
1
vote
0answers
25 views

smallest poisitive integer does not belong in floating point system F

A floating-point number system F is a subset of real numbers whose elements have the form: $x=\pm(m/\beta^t)\beta^e$ base (beta), precision (t), and exponent range (emin and emax). The mantissa m ...
0
votes
1answer
78 views

Convergence and limit of Muller sequence

The Muller sequence is given by the recursive definition: $U(n+1)=111-\frac{1130}{U(n)}+\frac{3000}{U(n)U(n-1)}$ with $U(0)=5.5$ and $U(1)=\frac{61}{11}$. This sequence is interesting in ...
1
vote
1answer
56 views

Entropy of floating number array

I am familiar with shanon's definition of entropy. $$ H(P) = - \sum_{i=1}^n p_i \cdot \log_2(\mathcal p_i) $$ I am today in the situation that I'd like to compute an entropy like function but for a ...
2
votes
4answers
103 views

How to extract fraction from a floating point number

I'm making some tests with float type (floating point number) with programming and in some of my tests I need to extract the fraction that originates the float value. Let $ x $ be a floating point ...
0
votes
0answers
78 views

Different SVD results in Matlab

my question relates to calculating SVD in Matlab. I have been reading a lot and somehow I have jumbled up all the facts. It would be great if you experts could get me to the right track. My task is ...
2
votes
2answers
105 views

Floating point arithmetic: $(x-2)^9$

This is taken from Trefethen and Bau, 13.3. Why is there a difference in accuracy between evaluating near 2 the expression $(x-2)^9$ and this expression: $$x^9 - 18x^8 + 144x^7 -672x^6 + 2016x^5 - ...
0
votes
1answer
220 views

Converting double precision IEEE 754 hex to base 10 with repeating decimals

The number is 0x4001 8CCC CCCC CCCC. So far I have the stored exponent as 1000000000 which equals ...
1
vote
0answers
33 views

Machine Floating Point Theorem

Completely stuck on this floating point question. Let $x \in \mathbb{R}$ have the following floating point representation: $$ x = (-1)^s[0.a_1a_2\dots a_ta_{t+1}\dots]\cdot \beta^e $$ [Where $\beta$ ...
0
votes
1answer
29 views

Finite binary representation of float number

How can I prove that float number $x$ has the finite binary representation if and only if it is written like that: $ x = m / 2^n $, where $m, n \in \mathbb N$. Should I consider something like 3 ...
2
votes
3answers
317 views

What is the maximum difference between two successive real numbers in the given floating point representation?

The following is a scheme for floating point number representation using 16 bits. Sign :- Bit 15 Exponent:-Bit 14-9 Mantissa :- Bit 8-0 Let $s, e,$ and $m$ be the numbers represented in binary in ...
6
votes
1answer
195 views

How to calculate floating point numbers?

Here are two locations in small memory: 0110 | 1111 1110 1101 0011 0111 | 0000 0110 1101 1001 Interpret locations 6 and 7 as an IEEE floating point number. ...
1
vote
5answers
69 views

Not clear on what we mean with numbers with infinite digits

I am confused on a rather simplistic question. 1/3 = 0.333333333333 to infinity. So it has infinite digits. How is it possible to multiply such a number with another one and get a finite number? 6/3 = ...
1
vote
3answers
40 views

How to interpret fractional number of bits of precision

In double-precision floating-point format there're effective $53$ bits of mantissa stored. This lets us estimate maximum number of decimal digits of precision available: ...
2
votes
1answer
111 views

Accurate floating-point linear interpolation

I want to perform a simple linear interpolation between $A$ and $B$ (which are binary floating-point values) using floating-point math with IEEE-754 rounding rules, as accurately as possible. Please ...
1
vote
1answer
65 views

Converting 16bit float to Base10 and vice versa

Hi! I have some difficulties understanding how I'm supposed to calculate this 16bit float to base10. This is something that is coming up on a test so I would be pleased to learn how this is supposed ...
5
votes
1answer
1k views

What is the most significant digit?

What is the most significant digit of $$0.00234$$ I have a problem of figuring out where it is $0$ or $2$.
0
votes
0answers
82 views

Overflow and underflow of a probability value

I am evaluating the probability that the minimum of a process is a above a a barrier $\log(H)$. The probability is given by $$P_i=1-\exp\left(-2\frac{(\log(H)-x)(\log(H)-x_b)}{\tau\sigma^2}\right).$$ ...
2
votes
1answer
134 views

Given the normalised floating point number system, calculate smallest possible value of y - x

First I present the problem and then my workings and thoughts: Given the normalized floating point number system $(\beta, t, L, U) = (10, 7, -6, 4)$ where $\beta$ is the base and $t$ is the ...
0
votes
3answers
98 views

Is this approach for testing orthogonality/parallelity of vectors wrong as I think?

In a math book four methods are written for testing parallelity/orthogonality of two vectors that are(notice $\vec v$ and $\vec w$ are approximations of vectors and we have x,y and z components of ...