Mathematical questions concerning floating point numbers, a finite approximation of the real numbers used in computing.

learn more… | top users | synonyms

0
votes
0answers
14 views

Whether assigning of single precision IEEE754 float to double is reversible?

Within scope of IEEE754 standard let's assign single precision variable s to double precision variable d and then assign d to single precision variable s'. Whether this operation is ...
0
votes
0answers
20 views

What should be the precision to uniquely recover the binary double float number from the decimal one?

Assume we store binary float number in IEEE 754 format to decimal format. Then we recover decimal to binary back. With regard to single floats - according to this article When a binary IEEE single ...
0
votes
1answer
60 views

Using IEEE floating point standard in math-software. [on hold]

For purposes of given math module (that performs some statistical computations) it's sufficient to store reals in IEEE754 floating point format. I'm considering to migrate to SQLLite as persistent ...
-1
votes
1answer
17 views

Getting precision of integer given a float precision point

Let's say I have the number 1234567 and I want to get the value 12.34567 with a precision point of ...
2
votes
2answers
173 views

How can I rearrange this logarithmic formula to be computer friendly?

I've had a look through the logarithmic identities on Wikipedia, but nothing fits the bill. Basically, I have a formula which shows how much more 'risky' one number is compared to another, where 0 = ...
0
votes
2answers
28 views

16.75 How to convert to floating point representation? [closed]

16.75 convert to base 2 floating point representation. Need help on formula, Thanks.
2
votes
1answer
51 views

Rounding unit vs Machine precision

I'm not sure if this question should be asked here... For a general floating point system defined using the tuple $(\beta, t, L, U)$, where $\beta$ is the base, $t$ is the number of bits in the ...
1
vote
1answer
28 views

How are floating-point numbers logarithmically distributed?

From what I remember from a lecture I had of a course I'm attending called "introduction to computational science", floating-point numbers are distributed logarithmically. What does it mean? And how ...
0
votes
2answers
23 views

Help with calculating relative error in approximation of x

So i have $$x=\displaystyle\sum_{k=1}^{\infty}2^{-k}+\displaystyle\sum_{k=0}^{\infty}2^{-6k-1}$$ and i need to calculate relative error when approximating above x in $$MARC-32 \ \dots ...
1
vote
1answer
25 views

Can we get negative variance when calculating it for a small dataset using a naive formula?

In Knuth's Volume 2 Seminumerical Algorithms, chapter 4.2.2 Accuracy of Floating Point Arithmetic, there's a statement: Novice programmers who calculate the standard deviation of some observations ...
1
vote
1answer
28 views

Modulo multiplicative inverse of floating numbers

I have a floating value $k$ and an integer $P$ I want to calculate $(\dfrac{k}{\sqrt5}) \mod P$ How do I calculate it? PS: I know how to calculate MMI (Modulo Multiplicative Inverse of integer ...
0
votes
0answers
12 views

Is there any difference between fixed point and decimal point?

Source: Introduction to Computers' 1999 Ed.1999 Edition Fixed point number 774.3675 is just a decimal number with a decimal point to show a fractional part 3675/10000. I see no difference in the ...
1
vote
0answers
26 views

How do we tell arithemetic addition or subtraction from floating point numbers?

Source: C++ for Engineers and Scientists, Gary J. Bronson Source: Programmable Logic Controllers: The Complete Guide to the Technology by Clarence T. Jones In the table $12345.67_{10}$ = ...
1
vote
1answer
38 views

What does leading $0s$ in a number in scientific notation mean?

Source: Computer Organization and Design: The Hardware/software Interface, David A. Patterson,John L. Hennessy It doesn't seem as the author is using leading $0$ like leading $1$ in a matrix. What ...
1
vote
0answers
26 views

If $1/2 ≤ p/q ≤ 2$ , then $p-q$ is representable exactly on the computer

I've found the following affirmation in an article. I've been thinking about it but I don't know the way to prove it: It is not hard to prove that if $p$ and $q$ are two of a computer’s ...
0
votes
1answer
109 views

can anyone please explain how 2.8 modulo 2 is 0.7999999999999998? [duplicate]

I am not a mathematician, and just started programming in javascript and wonders how 2.8 % 2 = 0.7999999999999998. Note: I know it is remainder operation. May be I forgot my school mathematics ...
0
votes
0answers
15 views

Given a base, a mantissa size and a range for the exponent, what's the largest number?

I was introduced to floating-point numbers week ago or so, and although I think I understood the basics, I'm still not sure how to apply them. A floating-point number is usually represented with the ...
0
votes
0answers
14 views

Simple computations on decimal floating point arithmetic

Can anyone please check my working on the computations of $2*0.3-0.6$ and $3*0.3-0.9$ on decimal floating point? My working: $2*0.3-0.6=(2.0*10^0*3.0*10^{-1})-6.0*10^{-1}=6.0*10^{-1}-6.0*10^{-1}=0$ ...
8
votes
4answers
2k views

Why is $3 \times 0.3 = 0.8999999999999999$ in floating point?

Can anyone please help me explain this fact? I tried to read some articles on the web about floating point but it is always a hard topic for me to understand. This is what I get from Python 3.3.0 ...
0
votes
2answers
32 views

Computing midpoint of an interval overflow

For computing the midpoint m of an interval $[a, b]$, which of the following two formulas is preferable in floating-point arithmetic? Why? When? (Hint: Devise examples for which the "midpoint" given ...
0
votes
0answers
22 views

What are 2*0.3-0.6 and 3*0.3-0.9 in decimal floating point numbers?

Investigate the values $a=2*0.3-0.6$ and $b=3*0.3-0.9$. What do we expect to get on a computer using decimal floating point numbers? My reason and doubt: It doesn't really mention how many degree ...
0
votes
1answer
56 views

Find original inputs x and ^y for a given product, possible or not? [closed]

387,381,625,547,900,583,936 is the product of this calculation 21 * 2^64. If I only have the product and the multiplier 2 (without the exponent) would it be possible to find the other inputs used to ...
4
votes
2answers
45 views

Equation demonstration

Let $\delta f \equiv \frac{\Delta f}{f}$ show that: $$\matrix{\delta(xy) &=& \delta x + \delta y\\ \delta(x/y) &=& \delta x - \delta y\\ \delta(x+y) &=& \frac{x}{x+y}{\delta ...
0
votes
1answer
46 views

Why do we want to keep mantissa as small as possible and exponent as large as possible?

Why do we want to keep mantissa as small as possible and exponent as large as possible in the floating point representation of numbers? What would happen if we make mantissa as big as possible? ...
0
votes
1answer
34 views

Need an Ambiguous example

Suppose I have a floating-point number with $m$ where $(m > 0)$ digits after the decimal point. Now if I want to round it up to $d$ where $(0 ≤ d < m)$ digits after the decimal point, sometimes ...
3
votes
1answer
36 views

Computing 4-digit float point arithmetic

Compute $f(π)$ using 4-digit float point arithmetic if $f(x) = \sqrt{x^2 - 8 - \sqrt{x} }$ Is my $x = 3.141$? and is $f(x) = 0.306$?
1
vote
1answer
34 views

Stable algorithm for computation of $\Phi(20)$, when $\displaystyle \Phi(x)=\frac{2}{\sqrt{\pi}}\sum_{k=0}^{\infty}(-1)^k\frac{x^{2k+1}}{k!(2k+1)}$

Let $\displaystyle \Phi(x)=\frac{2}{\sqrt{\pi}}\sum_{k=0}^{\infty}(-1)^k\frac{x^{2k+1}}{k!(2k+1)}$, i.e $\Phi$ is the MacLaurin series of the function $\displaystyle ...
0
votes
2answers
68 views

Proving that $a\le \text{fl}\left(\frac{a+b}{2}\right)\le b$

Suppose that $a$ and $b$ are some floating point numbers such that $a\lt b$. How can I show that $$a\le \text{fl}\left(\frac{a+b}{2}\right)\le b$$ specifically in IEEE standard floating point ...
0
votes
1answer
51 views

For which arguments in the range $0\le x\le \pi/4$ will $\cos x=(1-\sin^2x)^{1/2}$ fail to give good accuracy?

The question is In floating point system, consider using the trigonometric identity $\sin^2x+\cos^2x=1$ to compute $\cos x=(1-\sin^2x)^{1/2}$. For which arguments in the range $0\le x\le \pi/4$ ...
0
votes
1answer
52 views

One wants to determine $1/(2+\sqrt 3)^4$ having access to an approximate value of $\sqrt 3$.

Can someone help me with this question please: One wants to determine $1/(2+\sqrt 3)^4$ having access to an approximate value of $\sqrt 3$. Compare the relative errors on direct computation and on ...
3
votes
1answer
34 views

Formal theory on floating point numbers?

Is there a formal theory involving the set of floating point numbers? Like topological properties, analytic properties, etc. There's no abstract theory involving floating point set? I usually find a ...
0
votes
0answers
18 views

Floating point binary to half precision floating point

I am trying to convert to $16$ bit half precision floating point however I ran into a possible error and am unsure if a negative exponent is ok. I am trying to convert $0010011100010000$ I separate ...
2
votes
1answer
38 views

Explaining error in a floating-point calculation

I'm working through a numerical analysis text and came across this question. The function $f_{1}(x,\delta) = \cos(x + \delta) - \cos(x)$ can be transformed into another form, $f_{2}(x,\delta)$, ...
1
vote
1answer
47 views

The arctangent is a strange floating point number

I have 2 players in a game (Cod 4). I read X, Y, Z and store them in: ...
0
votes
1answer
29 views

How do I perform Gram-Schmidt on floating point vectors with epsilons in them?

Let $\epsilon$ be a small positive number such that $1+\epsilon$ and $3+2\epsilon$ are machine numbers but $3+2\epsilon + \epsilon^{2}$ is computed to be $3 + 2\epsilon $. Now, let the (classical) ...
0
votes
1answer
22 views

Floating Point and machine error

Let $a$ and $b$ be to real arbitrary real numbers, show that the relative error that you made by computing $a^2b$ with floating point arithmethic is bound to $5\epsilon + O(\epsilon^2)$, with ...
15
votes
3answers
219 views

Plotting $\left(1+\frac{1}{x^n}\right)^{x^n}$.

When I plot the following function, the graph behaves strangely: $$f(x) = \left(1+\frac{1}{x^{16}}\right)^{x^{16}}$$ While $\lim_{x\to +\infty} f(x) = e$ the graph starts to fade at $x \approx 6$. ...
1
vote
1answer
57 views

Two expressions using three-digit floating point arithmetic with rounding?

What is the result of evaluating the following two expressions using three-digit floating point arithmetic with rounding? $(113. + -111.) + 7.51$ $113. + (-111. + 7.51)$ $9.51$ and $10.0$ ...
1
vote
0answers
17 views

How to bound the input parameters to a chaotic function to obtain exact result in a finite precision setting?

While I was reading the paper entitled (http://dx.doi.org/10.1109/ISCAS.2003.1204947) Kocarev, Ljupco, and Zarko Tasev. "Public-key encryption based on Chebyshev maps." Circuits and Systems, ...
1
vote
3answers
38 views

Floating point overflow. Can this power equation be simplified?

I’m running into trouble with the following formula that we’re using in our software. $$ \frac{1 - x^{-y}}{1 - x^{-(y+1)}} $$ In certain cases, the value for $y$ is a relatively large number; ...
1
vote
2answers
101 views

Floating point modulus precision (high multiple)

While looking at implementing floating point modulus in double-precision on an x86 CPU, I found the FPREM instruction and proceeded to test the practical precision in some cases, one being ...
0
votes
0answers
10 views

What line-polygon clipping algorithm can I use to ensure that the resultant endpoints are always within the polygon?

I have a 2D plane, partitioned into n-sided, convex polygons. I'm using WRF's PNPOLY algorithm for polygon inclusion to ensure that a point belongs inside one and only one polygon. Is there an ...
1
vote
1answer
53 views

What is the largest floating point number a so that fl(100 + a) = 100?

What is the largest floating point number a so that fl(100 + a) = 100? Here is how float number is computed. $fl(a ⊙ b) = (a ⊙ b)(1 + δ)$. Where $|δ| ≤ ε$. Furthermore, $ε = 2^{-53}$. My ...
1
vote
0answers
49 views

If $0.2349$ is approximated to $0.2299$, what is number of significant figures in such approximation?

If $0.2349$ is approximated to $0.2299$, what is number of significant figures in such approximation? Let $x_T=0.2349$, $x_A=0.2299$, then absolute error = $|x_T-x_A|=0.0050=\frac{1}{2}\times ...
0
votes
0answers
108 views

floating point subtraction for binary numbers

Consider that I want to do a binary operation on the following floating point numbers: 0.35-0.62 I can reach the end but I can not figure out how the sign bit is ...
0
votes
1answer
61 views

Finding the mantissa from binary with floating point numbers?

Here is the example problem slide I am working with: I understand how to get the exponent, its just 2+128=130-127=3 I understand the first bit is the sign bit for positive or negative. I just ...
0
votes
0answers
15 views

Fixed point exponent faithfull rounding? does it make any sense?

i've a stupid question that just come up in my mind. Let's say we have a floating point number $x = (-1)^{s_x} 2^{e_x} 1.m_x$, let's say for some reason we need to divide the exponent by $2$, maybe ...
0
votes
1answer
13 views

Computer arithmetic - size of error.

I'm actually reading What Every Computer Scientist Should Know About Floating-Points Arithmetics and I have a little problem with understaing one thing. Here's a quote: Write $(b \otimes b) ...
1
vote
0answers
27 views

Error accumulation

Assume that $a_j\leq u$ and $p_j\in\{-1,+1\}$ for $j=1,2,\ldots,n$ and that $nu<1$ where $u:=2^{-t-1}$. Show that following is true: $$\prod^{n}_{j=1}(1+a_j)^{p_j}=1+\theta_n$$ where ...
0
votes
0answers
32 views

Longest string whose $\log$ probability can be represented

The problem starts out with a binary string of length $n$, so the probability of any random string in the set is $\cfrac{1}{2^n}$. The smallest positive floating point value that the system can ...