Subject: Re: Binary floating point division
Date: Sat, 24 Jan 1998 12:48:37 -0500
From: Alex Bogomolny
It's a known dictum that the output is at best as good as input. So the first part of Netscape's claim is a complete nonsense. The problem is of course in that they return raw result of a decimal operation executed in binary arithmetic.
Assume A and B are exact numbers while a and b are their respective approximations: |A - a| < e1 and |B - b| < e2. In your case, A = 11.8, B = 5. a and b are their approximations in, say, 64 bit binary. I do not know how numbers are represented in a 64 bit PC. I would assume that 1/5 (or, probably, 13 bits) are occupied by exponent, 1 goes for the sign, 50 remain for mantissa. It's safe to assume that both errors e1 and e2 are close to 2-50.
You want to estimate |A/B - a/b|. (A/B = 2.36 and a/b is what JS hands out to you.)
|A/B - a/b| = |b(A - a) - a(B - b)| / (Bb)
Approximately this is bounded from above by |A - a|/B + |B - b|A/B2 = 2-50*(.2 + 11.8/25) = 2-50*.672 Which is approximately .672*10-15. And this is only twice as large as JS gave you. The reason is that binary arithmetic might smartly use rounding instead of chopping. So that you'd get e1 = e2 = 2-49 instead of 2-50. Everything fits snugly.
A good calculator should probably detect such cases and try to make up for inaccuracy in the binary arithmetic.