Float data type precision

Janet Young jayoung at fhcrc.org
Wed Jun 27 03:13:56 EST 2001


> Input:
> Float_class : "Example"
> Value 0.1234567
> Value 1234567
> Value 1234567.1234567
> 
> Output:
> Float_class : "Example"
> Value    0.123457
> Value    1234567.000000
> Value    1234567.125000


My opinion - it would be nice but not essential to have more digits
available - I have some data which is positions on Jim Kent's UCSC genome
assemblies, so some of the positions get up to 100,000,000bp kind of sizes.
big numbers are getting rounded e.g. 135184853 gets turned into 1.35185e+08.

Janet


-------------------------------------------------------------------
                              Dr. Janet Young
Fred Hutchinson Cancer Research Center
1100 Fairview Avenue N.                    tel: (206) 667 1471
C3-168                                     fax: (206) 667 6524
P.O. Box 19024                             email: jayoung at fhcrc.org
Seattle, WA 98109-1024, USA.






More information about the Acedb mailing list