Android Question Calculating Numbers with 1 or 2 Decimals

[chrjak]

Hey Community

Everything my project is doing is adding 5.3 to 26.5.
In a toastmessage(26.5+5.3,...) it correctly shows 31.8
However doing it by code and letting the numbers read in by a file (I checked them during debugging - it really adds this numbers like they are above) results in something like 31.8000000000132 or 31.7999999999132.

I tested my A** off to solve this.

Rounding with BigDecimals or NumberFormat does not change the number at all.

I tried different types:
Everything as Strings
... as Floats
... as Shorts
... as Doubles

Then i tried doing all the calculations not by code but by using BigDecimals.

Always the same result.. Why!?!?!? I don't get it

I just want to calculate money values with a maximum of 2 decimals and it becomes difficult like coding a science app.

____
I can send the project in private chat

 

[Diceman]

Perhaps it is the debugger that is the problem? The debugger may be rounding the number for display purposes and the number is actually stored correctly in the variable?

What happens if you create an if statement like:

private Num as Double
Num = 5.3 + 26.5
log("Num=" & Num)
if num=31.8 then
  log("success")
else
  log("failed")
end if

 

[gkoehn]

Is there any way you can convert your numbers to integer then calculate then convert the result back to a float? Would that help?

 

[MarkusR]

i found a lib for it
https://www.b4x.com/android/help/bignumbers.html

    Dim a As BigDecimal
    a.Initialize4(53,1)
    Dim b As BigDecimal
    b.Initialize4(265,1)
    Dim c As BigDecimal
    c.Initialize5(a.Add(b))
    c.SetScale(2)
    Log(c.ToPlainString)

https://docs.oracle.com/javase/10/docs/api/java/math/package-summary.html

 

[DonManfred]

results in something like 31.8000000000132

use numberformat to format a number.

 

[Diceman]

use numberformat to format a number.

Chrjak needs to keep in mind that NumberFormat() and NumberFormat2() will work well for displaying numbers because it converts the number to a string with the specified # of decimal places.

If chrjak was concerned the variables were losing precision when the values were being stored internally he needs to use the BCD library mentioned above.
I wrote a simple program that demonstrates the "problem". It goes through a loop adding 5.3 + 26.5 and accumulating the rounding error. I did this to determine if the rounding error was introduced when the number was being displayed, or if the rounding error was being stored in the variable internally.

The rounding is being stored in the variable internally. But it is quite small, around 3E-15. If chrjak is doing looping and creating totals then the BCD library mentioned in #4 by MarkusR should help. If on the other hand he doesn't have a loop then Don's suggestion of using NumberFormat() and NumberFormat2() will solve the problem when the number is displayed. All programming languages have this type of rounding error unless the programmer resorts to using BCD which is considerably slower.

Sub TestAdd(aMax As Int)
    Private Num, Total As Double
    Total = 0
    For i=1 To aMax
        Num = 5.3 + 26.5
        Total = Total + Num
        Log($"i=${i}  Num=${Num} Total=${Total} Avg=${Total/i} Diff=${31.8-(Total/i)}"$)
    Next
End Sub



Log:
-----
i=1  Num=31.8 Total=31.8 Avg=31.8 Diff=0.0
i=2  Num=31.8 Total=63.6 Avg=31.8 Diff=0.0
i=3  Num=31.8 Total=95.4 Avg=31.8 Diff=0.0
i=4  Num=31.8 Total=127.2 Avg=31.8 Diff=0.0
i=5  Num=31.8 Total=159.0 Avg=31.8 Diff=0.0
i=6  Num=31.8 Total=190.8 Avg=31.8 Diff=0.0
i=7  Num=31.8 Total=222.60000000000002 Avg=31.800000000000004 Diff=-3.552713678800501E-15
i=8  Num=31.8 Total=254.40000000000003 Avg=31.800000000000004 Diff=-3.552713678800501E-15
i=9  Num=31.8 Total=286.20000000000005 Avg=31.800000000000004 Diff=-3.552713678800501E-15
i=10  Num=31.8 Total=318.00000000000006 Avg=31.800000000000004 Diff=-3.552713678800501E-15
i=11  Num=31.8 Total=349.80000000000007 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=12  Num=31.8 Total=381.6000000000001 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=13  Num=31.8 Total=413.4000000000001 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=14  Num=31.8 Total=445.2000000000001 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=15  Num=31.8 Total=477.0000000000001 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=16  Num=31.8 Total=508.8000000000001 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=17  Num=31.8 Total=540.6000000000001 Avg=31.800000000000008 Diff=-7.105427357601002E-15
i=18  Num=31.8 Total=572.4000000000001 Avg=31.800000000000004 Diff=-3.552713678800501E-15
i=19  Num=31.8 Total=604.2 Avg=31.8 Diff=0.0
i=20  Num=31.8 Total=636.0 Avg=31.8 Diff=0.0
i=21  Num=31.8 Total=667.8 Avg=31.799999999999997 Diff=3.552713678800501E-15
i=22  Num=31.8 Total=699.5999999999999 Avg=31.799999999999997 Diff=3.552713678800501E-15
...
...
i=964  Num=31.8 Total=30655.19999999952 Avg=31.799999999999503 Diff=4.973799150320701E-13
i=965  Num=31.8 Total=30686.99999999952 Avg=31.799999999999503 Diff=4.973799150320701E-13
i=966  Num=31.8 Total=30718.79999999952 Avg=31.799999999999503 Diff=4.973799150320701E-13
i=967  Num=31.8 Total=30750.59999999952 Avg=31.799999999999503 Diff=4.973799150320701E-13
i=997  Num=31.8 Total=31704.599999999497 Avg=31.799999999999496 Diff=5.044853423896711E-13
i=998  Num=31.8 Total=31736.399999999496 Avg=31.799999999999496 Diff=5.044853423896711E-13
i=999  Num=31.8 Total=31768.199999999495 Avg=31.799999999999496 Diff=5.044853423896711E-13
i=1000  Num=31.8 Total=31799.999999999494 Avg=31.799999999999493 Diff=5.080380560684716E-13