Float data type precision

Mike Dunn mike.dunn at amd.com
Tue Jun 26 04:55:05 EST 2001


I'm running ACEDB Version 4_7l and am getting rounding errors with
large Float values.  Here is an example class, the data I enter in the
database, and what the db returns to me

Class model:
?Float_class Title UNIQUE Text
             Value Float

Input:
Float_class : "Example"
Value 0.1234567
Value 1234567
Value 1234567.1234567

Output:
Float_class : "Example"
Value	 0.123457
Value	 1234567.000000
Value	 1234567.125000

Is any of this rounding error reduced in later versions of acedb?  Is
there anyway to get more precision?  I'm using aceperl, so one
workaround is to store the large floats as text, then let perl sort it
out, but I'd rather find a way to keep the data in the db as Float.

Regards,
Mike Dunn
------------------------------------------------------------------
Mike Dunn                            CMD / CPG 
Advanced Micro Devices               email: Mike.Dunn at amd.com
5204 E. Ben White Blvd.              phone: (512) 602-5231
Austin, TX 78741                     fax:   (512) 602-5155
(Mail Stop 521)                      pager: (512) 622-6413
------------------------------------------------------------------





More information about the Acedb mailing list