Subject: Re: Binary floating point division
Date: Sat, 24 Jan 1998 14:39:05 -0500
From: Alex Bogomolny

Bernd, hello:

You are right about one possible remedy: evaluating the conversion error from decimal to binary and chopping it off. There might be a simpler solution:

Convert you numbers to integers or long integers, as the case may be. In integer arithmetic Netscape behaves much better. For example, 118/50 = 2.36. What it takes is storing numbers as pairs - (power of ten, integer) so that 11.8 = (-1, 118).

The expression "integer arithmetic" is not probably accurate. But, if it works, who cares. I am not familiar with the system to give a satisfactory explanation. There could be several reasons.

Best regards,
Alexander Bogomolny

|Reply| |Up|

Copyright © 1996-2017 Alexander Bogomolny


Search by google: