Mathematical questions concerning floating point numbers, a finite approximation of the real numbers used in computing.

learn more… | top users | synonyms

1
vote
0answers
9 views

Calculating range and eps-machine of floating-point system

Suppose I have a 5-bit floating point system with a 3 bit exponent with radix $\beta = 2$. What is the range and $\epsilon_{machine}$ of this system? I know that I can write numbers as: $$sign ...
1
vote
1answer
52 views

Converting 0.1 to binary 64 bit double

I want to convert the decimal number 0.1 to binary 64 bit double. So I do it like that: $$ 0.1_{10} = 0.00011001100110011001100110011001100110011001100110011001100110... \times 2^0 $$ Represent it ...
0
votes
1answer
33 views

How to find the difference for an infinite fraction

Suppose I have the number 0.101 in binary. If I want to round it to 2 places after the radix point using the algorithm rounding to the nearest I can easily find the ...
0
votes
1answer
20 views

Rounding to nearest

I have the number 0.101 in binary. I want to round it to 2 places after the radix point using the algorithm ...
0
votes
0answers
33 views

How do I round this binary number to the nearest even

I have this binary representation of 0.1: 0.00011001100110011001100110011001100110011001100110011001100110 I need to round it ...
0
votes
0answers
36 views

Automatic curve fitting to find order of an algorithm?

I'm a newbies in mathematics. I'm looking for an automatic best curve fitting function to find the order of an algorithm. I would like to know if it does exists a math library function that would ...
0
votes
0answers
15 views

Whether assigning of single precision IEEE754 float to double is reversible?

Within scope of IEEE754 standard let's assign single precision variable s to double precision variable d and then assign d to single precision variable s'. Whether this operation is ...
0
votes
0answers
20 views

What should be the precision to uniquely recover the binary double float number from the decimal one?

Assume we store binary float number in IEEE 754 format to decimal format. Then we recover decimal to binary back. With regard to single floats - according to this article When a binary IEEE single ...
0
votes
1answer
60 views

Using IEEE floating point standard in math-software. [closed]

For purposes of given math module (that performs some statistical computations) it's sufficient to store reals in IEEE754 floating point format. I'm considering to migrate to SQLLite as persistent ...
-1
votes
1answer
18 views

Getting precision of integer given a float precision point

Let's say I have the number 1234567 and I want to get the value 12.34567 with a precision point of ...
2
votes
2answers
176 views

How can I rearrange this logarithmic formula to be computer friendly?

I've had a look through the logarithmic identities on Wikipedia, but nothing fits the bill. Basically, I have a formula which shows how much more 'risky' one number is compared to another, where 0 = ...
0
votes
2answers
28 views

16.75 How to convert to floating point representation? [closed]

16.75 convert to base 2 floating point representation. Need help on formula, Thanks.
2
votes
1answer
54 views

Rounding unit vs Machine precision

I'm not sure if this question should be asked here... For a general floating point system defined using the tuple $(\beta, t, L, U)$, where $\beta$ is the base, $t$ is the number of bits in the ...
1
vote
1answer
28 views

How are floating-point numbers logarithmically distributed?

From what I remember from a lecture I had of a course I'm attending called "introduction to computational science", floating-point numbers are distributed logarithmically. What does it mean? And how ...
0
votes
2answers
25 views

Help with calculating relative error in approximation of x

So i have $$x=\displaystyle\sum_{k=1}^{\infty}2^{-k}+\displaystyle\sum_{k=0}^{\infty}2^{-6k-1}$$ and i need to calculate relative error when approximating above x in $$MARC-32 \ \dots ...
1
vote
1answer
25 views

Can we get negative variance when calculating it for a small dataset using a naive formula?

In Knuth's Volume 2 Seminumerical Algorithms, chapter 4.2.2 Accuracy of Floating Point Arithmetic, there's a statement: Novice programmers who calculate the standard deviation of some observations ...
1
vote
1answer
29 views

Modulo multiplicative inverse of floating numbers

I have a floating value $k$ and an integer $P$ I want to calculate $(\dfrac{k}{\sqrt5}) \mod P$ How do I calculate it? PS: I know how to calculate MMI (Modulo Multiplicative Inverse of integer ...
0
votes
0answers
15 views

Is there any difference between fixed point and decimal point?

Source: Introduction to Computers' 1999 Ed.1999 Edition Fixed point number 774.3675 is just a decimal number with a decimal point to show a fractional part 3675/10000. I see no difference in the ...
1
vote
0answers
26 views

How do we tell arithemetic addition or subtraction from floating point numbers?

Source: C++ for Engineers and Scientists, Gary J. Bronson Source: Programmable Logic Controllers: The Complete Guide to the Technology by Clarence T. Jones In the table $12345.67_{10}$ = ...
1
vote
1answer
38 views

What does leading $0s$ in a number in scientific notation mean?

Source: Computer Organization and Design: The Hardware/software Interface, David A. Patterson,John L. Hennessy It doesn't seem as the author is using leading $0$ like leading $1$ in a matrix. What ...
1
vote
0answers
26 views

If $1/2 ≤ p/q ≤ 2$ , then $p-q$ is representable exactly on the computer

I've found the following affirmation in an article. I've been thinking about it but I don't know the way to prove it: It is not hard to prove that if $p$ and $q$ are two of a computer’s ...
0
votes
1answer
111 views

can anyone please explain how 2.8 modulo 2 is 0.7999999999999998? [duplicate]

I am not a mathematician, and just started programming in javascript and wonders how 2.8 % 2 = 0.7999999999999998. Note: I know it is remainder operation. May be I forgot my school mathematics ...
0
votes
0answers
16 views

Given a base, a mantissa size and a range for the exponent, what's the largest number?

I was introduced to floating-point numbers week ago or so, and although I think I understood the basics, I'm still not sure how to apply them. A floating-point number is usually represented with the ...
0
votes
0answers
14 views

Simple computations on decimal floating point arithmetic

Can anyone please check my working on the computations of $2*0.3-0.6$ and $3*0.3-0.9$ on decimal floating point? My working: $2*0.3-0.6=(2.0*10^0*3.0*10^{-1})-6.0*10^{-1}=6.0*10^{-1}-6.0*10^{-1}=0$ ...
8
votes
4answers
2k views

Why is $3 \times 0.3 = 0.8999999999999999$ in floating point?

Can anyone please help me explain this fact? I tried to read some articles on the web about floating point but it is always a hard topic for me to understand. This is what I get from Python 3.3.0 ...
1
vote
2answers
34 views

Computing midpoint of an interval overflow

For computing the midpoint m of an interval $[a, b]$, which of the following two formulas is preferable in floating-point arithmetic? Why? When? (Hint: Devise examples for which the "midpoint" given ...
0
votes
0answers
25 views

What are 2*0.3-0.6 and 3*0.3-0.9 in decimal floating point numbers?

Investigate the values $a=2*0.3-0.6$ and $b=3*0.3-0.9$. What do we expect to get on a computer using decimal floating point numbers? My reason and doubt: It doesn't really mention how many degree ...
0
votes
1answer
57 views

Find original inputs x and ^y for a given product, possible or not? [closed]

387,381,625,547,900,583,936 is the product of this calculation 21 * 2^64. If I only have the product and the multiplier 2 (without the exponent) would it be possible to find the other inputs used to ...
4
votes
2answers
45 views

Equation demonstration

Let $\delta f \equiv \frac{\Delta f}{f}$ show that: $$\matrix{\delta(xy) &=& \delta x + \delta y\\ \delta(x/y) &=& \delta x - \delta y\\ \delta(x+y) &=& \frac{x}{x+y}{\delta ...
0
votes
1answer
48 views

Why do we want to keep mantissa as small as possible and exponent as large as possible?

Why do we want to keep mantissa as small as possible and exponent as large as possible in the floating point representation of numbers? What would happen if we make mantissa as big as possible? ...
0
votes
1answer
35 views

Need an Ambiguous example

Suppose I have a floating-point number with $m$ where $(m > 0)$ digits after the decimal point. Now if I want to round it up to $d$ where $(0 ≤ d < m)$ digits after the decimal point, sometimes ...
3
votes
1answer
38 views

Computing 4-digit float point arithmetic

Compute $f(π)$ using 4-digit float point arithmetic if $f(x) = \sqrt{x^2 - 8 - \sqrt{x} }$ Is my $x = 3.141$? and is $f(x) = 0.306$?
1
vote
1answer
35 views

Stable algorithm for computation of $\Phi(20)$, when $\displaystyle \Phi(x)=\frac{2}{\sqrt{\pi}}\sum_{k=0}^{\infty}(-1)^k\frac{x^{2k+1}}{k!(2k+1)}$

Let $\displaystyle \Phi(x)=\frac{2}{\sqrt{\pi}}\sum_{k=0}^{\infty}(-1)^k\frac{x^{2k+1}}{k!(2k+1)}$, i.e $\Phi$ is the MacLaurin series of the function $\displaystyle ...
0
votes
2answers
69 views

Proving that $a\le \text{fl}\left(\frac{a+b}{2}\right)\le b$

Suppose that $a$ and $b$ are some floating point numbers such that $a\lt b$. How can I show that $$a\le \text{fl}\left(\frac{a+b}{2}\right)\le b$$ specifically in IEEE standard floating point ...
0
votes
1answer
51 views

For which arguments in the range $0\le x\le \pi/4$ will $\cos x=(1-\sin^2x)^{1/2}$ fail to give good accuracy?

The question is In floating point system, consider using the trigonometric identity $\sin^2x+\cos^2x=1$ to compute $\cos x=(1-\sin^2x)^{1/2}$. For which arguments in the range $0\le x\le \pi/4$ ...
0
votes
1answer
52 views

One wants to determine $1/(2+\sqrt 3)^4$ having access to an approximate value of $\sqrt 3$.

Can someone help me with this question please: One wants to determine $1/(2+\sqrt 3)^4$ having access to an approximate value of $\sqrt 3$. Compare the relative errors on direct computation and on ...
3
votes
1answer
36 views

Formal theory on floating point numbers?

Is there a formal theory involving the set of floating point numbers? Like topological properties, analytic properties, etc. There's no abstract theory involving floating point set? I usually find a ...
0
votes
0answers
20 views

Floating point binary to half precision floating point

I am trying to convert to $16$ bit half precision floating point however I ran into a possible error and am unsure if a negative exponent is ok. I am trying to convert $0010011100010000$ I separate ...
2
votes
1answer
39 views

Explaining error in a floating-point calculation

I'm working through a numerical analysis text and came across this question. The function $f_{1}(x,\delta) = \cos(x + \delta) - \cos(x)$ can be transformed into another form, $f_{2}(x,\delta)$, ...
1
vote
1answer
47 views

The arctangent is a strange floating point number

I have 2 players in a game (Cod 4). I read X, Y, Z and store them in: ...
0
votes
1answer
31 views

How do I perform Gram-Schmidt on floating point vectors with epsilons in them?

Let $\epsilon$ be a small positive number such that $1+\epsilon$ and $3+2\epsilon$ are machine numbers but $3+2\epsilon + \epsilon^{2}$ is computed to be $3 + 2\epsilon $. Now, let the (classical) ...
0
votes
1answer
23 views

Floating Point and machine error

Let $a$ and $b$ be to real arbitrary real numbers, show that the relative error that you made by computing $a^2b$ with floating point arithmethic is bound to $5\epsilon + O(\epsilon^2)$, with ...
15
votes
3answers
223 views

Plotting $\left(1+\frac{1}{x^n}\right)^{x^n}$.

When I plot the following function, the graph behaves strangely: $$f(x) = \left(1+\frac{1}{x^{16}}\right)^{x^{16}}$$ While $\lim_{x\to +\infty} f(x) = e$ the graph starts to fade at $x \approx 6$. ...
1
vote
1answer
64 views

Two expressions using three-digit floating point arithmetic with rounding?

What is the result of evaluating the following two expressions using three-digit floating point arithmetic with rounding? $(113. + -111.) + 7.51$ $113. + (-111. + 7.51)$ $9.51$ and $10.0$ ...
1
vote
0answers
17 views

How to bound the input parameters to a chaotic function to obtain exact result in a finite precision setting?

While I was reading the paper entitled (http://dx.doi.org/10.1109/ISCAS.2003.1204947) Kocarev, Ljupco, and Zarko Tasev. "Public-key encryption based on Chebyshev maps." Circuits and Systems, ...
1
vote
3answers
39 views

Floating point overflow. Can this power equation be simplified?

I’m running into trouble with the following formula that we’re using in our software. $$ \frac{1 - x^{-y}}{1 - x^{-(y+1)}} $$ In certain cases, the value for $y$ is a relatively large number; ...
1
vote
2answers
103 views

Floating point modulus precision (high multiple)

While looking at implementing floating point modulus in double-precision on an x86 CPU, I found the FPREM instruction and proceeded to test the practical precision in some cases, one being ...
0
votes
0answers
10 views

What line-polygon clipping algorithm can I use to ensure that the resultant endpoints are always within the polygon?

I have a 2D plane, partitioned into n-sided, convex polygons. I'm using WRF's PNPOLY algorithm for polygon inclusion to ensure that a point belongs inside one and only one polygon. Is there an ...
1
vote
1answer
53 views

What is the largest floating point number a so that fl(100 + a) = 100?

What is the largest floating point number a so that fl(100 + a) = 100? Here is how float number is computed. $fl(a ⊙ b) = (a ⊙ b)(1 + δ)$. Where $|δ| ≤ ε$. Furthermore, $ε = 2^{-53}$. My ...
1
vote
0answers
49 views

If $0.2349$ is approximated to $0.2299$, what is number of significant figures in such approximation?

If $0.2349$ is approximated to $0.2299$, what is number of significant figures in such approximation? Let $x_T=0.2349$, $x_A=0.2299$, then absolute error = $|x_T-x_A|=0.0050=\frac{1}{2}\times ...