Subject: Binary floating point division
Date: Sat, 24 Jan 1998 00:44:52 +0100
From: Bernd Liebermann

Hi Alexander,

I've just been checking out the floating point base N conversion part of your really informative and great site. I found it after performing a Hotbot search as I'm looking for help with a strange behaviour of Netscape's JavaScript interpreter. I encountered this problem programming a kind of scientific JavaScript calculator: Netscape 4.04 returns 2.3600000000000003 for 11.8/5 - instead of simply 2.36. All other applications and MSIE's interpreter return the correct result. I assumed this to be a bug in Netscape and announced it in comp.lang.javascript. What I received as replies was: "That's not a bug! rather Netscape returns results of arithmetic expressions with a degree of accuracy that the calculation as itself doesn't have. Besides this, there's some unavoidable error in converting base 10 floating point values to binary ones". But all the answers on my problem couldn't take away my feeling that that '3' at the end of the fractional part of the number is in any way suspect. It's still sort of enigmatic to me where it comes from, and I can't help myself to believe there's an error in the algorithm that Netscape's JS interpereter uses.

I would appreciate it a lot if you could help me with that problem.

Thanks

Bernd Liebermann

--

Bernd Liebermann ...

Web Site Design and Management

... Marburger Str. 9 -- D-60487 Frankfurt -- +49-69-702498

|Reply| |Previous| |Next| |Down| |Exchange index| |Contents| |Store|

Copyright © 1996-2017 Alexander Bogomolny

 61211761

Search by google: