Float data type precision

Ed Griffiths edgrif at sanger.ac.uk
Tue Jun 26 07:55:32 EST 2001


Mike,

If I understand correctly what I've read about the IEEE standard for floating
point numbers, it is possible to represent up to 9 digits without rounding
error. BUT the ISO-C standard only requires compilers to support up to 6
significant digits (see FLT_DIG in the system header float.h). Jean has some
code to make sure that acedb will maintain precision on floats to this level of
accuracy which I think he is putting into the code any day now (is that correct
Jean ?).

In other words we cannot support your example below without introducing doubles
into acedb. If you feel this is a requirement then post back and perhaps we can
generate a discussion about this.

> Input:
> Float_class : "Example"
> Value 0.1234567
> Value 1234567
> Value 1234567.1234567
> 
> Output:
> Float_class : "Example"
> Value    0.123457
> Value    1234567.000000
> Value    1234567.125000

cheers Ed

 ------------------------------------------------------------------------
| Ed Griffiths, Acedb development, Informatics Group,                    |
|               The Sanger Centre, Wellcome Trust Genome Campus,         |
|               Hinxton, Cambridge CB10 1SA, UK                          |
|                                                                        |
| email: edgrif at sanger.ac.uk  Tel: +44-1223-494780  Fax: +44-1223-494919 |
 ------------------------------------------------------------------------





More information about the Acedb mailing list