|
|
|
|
|
|
|
|
|
Date: Fri, 16 Aug 96 20:01:00
From: Alex Bogomolny
Dear Matt,
please feel free to use my calculator any way you think fit. The behavior you describe is due to the limited size of the number representation used by JavaScript built-in functions. I look at the numbers you enter in the edit controls as 'long integers' that are most likely to be represented by 8 bytes or 64 bits. If a number requires more than that it will be truncated during its conversion from a string into an integer. Since truncated will be the most significant digits you not only lose accuracy but actually the meaning of the number.
To boot, the first bit is used to represent the sign - 1 for minus and 0 for plus. Which leaves us with 63 bits. This would imply that the largest hexadecimal integer correctly handled by the calculator is 0x7FFFFFFF which is 2147483647 decimal. I might have used what is called unsigned long integers which would have given 64 bit long numbers with the maximum of 0xFFFFFFFF and 4294967295 for hexadecimal and decimal representations respectively.
There is definitely a way to carry math operations with more precision. This requires a more substantial programming effort. The way the conversion is carried between different bases is described in most introductory books on Numerical Analysis.
Sincerely,
Alex
Copyright © 1996-2008 Alexander Bogomolny
| 29397586 |  |
|