Subject: Re: Binary floating point division
Date: Sat, 24 Jan 1998 12:48:37 -0500
From: Alex Bogomolny

Bernd, hello:

It's a known dictum that the output is at best as good as input. So the first part of Netscape's claim is a complete nonsense. The problem is of course in that they return raw result of a decimal operation executed in binary arithmetic.

Assume A and B are exact numbers while a and b are their respective approximations: |A - a| < e1 and |B - b| < e2. In your case, A = 11.8, B = 5. a and b are their approximations in, say, 64 bit binary. I do not know how numbers are represented in a 64 bit PC. I would assume that 1/5 (or, probably, 13 bits) are occupied by exponent, 1 goes for the sign, 50 remain for mantissa. It's safe to assume that both errors e1 and e2 are close to 2-50.

You want to estimate |A/B - a/b|. (A/B = 2.36 and a/b is what JS hands out to you.)

|A/B - a/b| = |b(A - a) - a(B - b)| / (Bb)

Approximately this is bounded from above by |A - a|/B + |B - b|A/B2 = 2-50*(.2 + 11.8/25) = 2-50*.672 Which is approximately .672*10-15. And this is only twice as large as JS gave you. The reason is that binary arithmetic might smartly use rounding instead of chopping. So that you'd get e1 = e2 = 2-49 instead of 2-50. Everything fits snugly.

A good calculator should probably detect such cases and try to make up for inaccuracy in the binary arithmetic.

Best regards,
Alexander Bogomolny

|Reply| |Up| |Down| |Exchange index| |Contents| |Store|

Copyright © 1996-2018 Alexander Bogomolny [an error occurred while processing this directive]
[an error occurred while processing this directive]