Floating point rounding error in Java

Issath Sesni
3 min readMay 17, 2021

--

Lets take an example to get the idea of the problem.

for(double d = 10; d != 0; d = d - 0.1){
System.out.println(d);
}

When we run the code, the expected output is something like this. i.e loop will be executed 100 times and values will be printed from 10.0 to 0.0. But the actual output is given below.

10.0
9.9
9.799999
9.699999
9.599998
9.499998
9.399998
9.299997
9.199997
9.099997
8.999996
8.899996
8.799995
8.699995
8.599995
8.499994
8.399994
8.2999935
8.199993
8.099993
7.999993
7.899993
7.799993
7.699993
7.599993
7.4999933
7.3999934
7.2999935
7.1999936
7.0999937
6.999994
6.899994
6.799994
6.699994
6.599994
6.4999943
6.3999944
6.2999945
6.1999946
6.0999947
5.9999948
5.899995
5.799995
5.699995
5.599995
5.499995
5.3999953
5.2999954
5.1999955
5.0999956
4.9999957
4.899996
4.799996
4.699996
4.599996
4.499996
4.3999963
4.2999964
4.1999965
4.0999966
3.9999967
3.8999968
3.7999969
3.699997
3.599997
3.4999971
3.3999972
3.2999973
3.1999974
3.0999975
2.9999976
2.8999977
2.7999978
2.699998
2.599998
2.499998
2.3999982
2.2999983
2.1999984
2.0999985
1.9999985
1.8999984
1.7999984
1.6999984
1.5999984
1.4999983
1.3999983
1.2999983
1.1999983
1.0999982
0.9999982
0.8999982
0.79999816
0.69999814
0.5999981
0.49999812
0.39999813
0.29999813
0.19999814
0.09999814
-1.861155E-6

Output is continuously decreasing (Infinite loop). This is happened because of Computers use IEEE 754 standard to represent floating point numbers. Now lets look at IEEE 754 standard.

IEEE 754 Standard

It uses 3 components.

  1. Sign bit :- 0 for positive, 1 for negative
  2. Exponent :- Represents in 2^n number of bits.
  3. Mantissa :- Binary representation of the scientific notation for base 2 numbers. We will take the bits after decimal point.
Double precision floating point number
Long double precision floating point number

Now lets look at an example 9.1 and represent it in IEEE 754 standard.

But the computer gets different IEEE 754 representation below.

0 | 10000010 | 00100011001100110011010

If the mantissa has more than 23 bits and round up to the upper limit. There computer will check 24th bit and add that value to 23rd bit. If 24th bit is 0, then no changes. If it is 1, rounding error step in, this will affect the value when we are converting back to decimal.

Lets convert this value back to decimal and check the differences.

2⁻³x1 + 2⁻⁷x1 + 2⁻8x1 + 2⁻11x1 + 2⁻¹²x1 + 2⁻¹⁵x1 + 2⁻¹⁶x1 + 2⁻¹⁹x1 + 2⁻²⁰x1 + 2⁻22x1
=0.137500048

We have to add 1 to this, cause we skip 1 in scientific representation. So,

1.137500048 * 2^3 = 9.10000038

That’s why we got output like that in our coding example above. In order to solve this problem, Developers use BigDecimal data type.

That’s all about this chapter. We will look BigDecimal in my another article. If I missed anything, drop in comment section.

References

--

--

Issath Sesni
Issath Sesni

No responses yet