Android Question Rounding decimal point number

aeric

Well-Known Member
Licensed User
Refer to the discussion earlier, how can I get 4.73 if I want convert 4.725 to 2 decimal place?

Rounding Numbers: Round2 or Numberformat2

Here is what I tried:
B4X:
    Dim dblValue As Double
        
    dblValue = 4.704
    For i = 0 To 9
        dblValue = NumberFormat(dblValue, 0, 3)
        Log("Number: " & dblValue)
        Log("NumberFormat(" & dblValue & ", 1, 2) = "  & NumberFormat(dblValue, 1, 2))
        Log("Round2(" & dblValue & ", 2) = "  & Round2(dblValue, 2))
        Log("-------------------------------------------------------")
        dblValue = dblValue + 0.01
    Next
    
    dblValue = 4.705
    For i = 0 To 9       
        dblValue = NumberFormat(dblValue, 0, 3)
        Log("Number: " & dblValue)
        Log("NumberFormat(" & dblValue & ", 1, 2) = "  & NumberFormat(dblValue, 1, 2))
        Log("Round2(" & dblValue & ", 2) = "  & Round2(dblValue, 2))
        Log("-------------------------------------------------------")
        dblValue = dblValue + 0.01
    Next
        
    dblValue = 4.706
    For i = 0 To 9
        dblValue = NumberFormat(dblValue, 0, 3)
        Log("Number: " & dblValue)
        Log("NumberFormat(" & dblValue & ", 1, 2) = "  & NumberFormat(dblValue, 1, 2))
        Log("Round2(" & dblValue & ", 2) = "  & Round2(dblValue, 2))
        Log("-------------------------------------------------------")
        dblValue = dblValue + 0.01
    Next
    
    Log("Extra")
    Log("-------------------------------------------------------")
    dblValue = 4.80
    Log("Number: " & dblValue)
    dblValue = NumberFormat(dblValue, 1, 2)
    Log("NumberFormat(" & dblValue & ", 1, 2) = "  & NumberFormat(dblValue, 1, 2))
    Log("NumberFormat2(" & dblValue & ", 1, 2, 2, False) = "  & NumberFormat2(dblValue, 1, 2, 2, False))
    Log("Round(" & dblValue & ") = "  & Round(dblValue))
    Log("Round2(" & dblValue & ", 1) = "  & Round2(dblValue, 1))
    Log("Round2(" & dblValue & ", 2) = "  & Round2(dblValue, 2))
    Log("-------------------------------------------------------")
    Log("Extra 2")
    Log("-------------------------------------------------------")
    dblValue = 4.725
    Log("Number: " & dblValue)
    dblValue = NumberFormat(dblValue, 1, 3)
    Log("NumberFormat(" & dblValue & ", 0, 2) = "  & NumberFormat(dblValue, 0, 2))
    Log("NumberFormat(" & dblValue & ", 1, 2) = "  & NumberFormat(dblValue, 1, 2))
    Log("NumberFormat2(" & dblValue & ", 0, 2, 2, False) = "  & NumberFormat2(dblValue, 0, 2, 2, False))
    Log("NumberFormat2(" & dblValue & ", 1, 2, 2, False) = "  & NumberFormat2(dblValue, 1, 2, 2, False))
    Log("Round(" & dblValue & ") = "  & Round(dblValue))
    Log("Round2(" & dblValue & ", 1) = "  & Round2(dblValue, 1))
    Log("Round2(" & dblValue & ", 2) = "  & Round2(dblValue, 2))
    Log("Ceil(" & dblValue & ") = "  & Ceil(dblValue))
    Log("Floor(" & dblValue & ") = "  & Floor(dblValue))
Results:
B4X:
Number: 4.704
NumberFormat(4.704, 1, 2) = 4.7
Round2(4.704, 2) = 4.7
-------------------------------------------------------
Number: 4.714
NumberFormat(4.714, 1, 2) = 4.71
Round2(4.714, 2) = 4.71
-------------------------------------------------------
Number: 4.724
NumberFormat(4.724, 1, 2) = 4.72
Round2(4.724, 2) = 4.72
-------------------------------------------------------
Number: 4.734
NumberFormat(4.734, 1, 2) = 4.73
Round2(4.734, 2) = 4.73
-------------------------------------------------------
Number: 4.744
NumberFormat(4.744, 1, 2) = 4.74
Round2(4.744, 2) = 4.74
-------------------------------------------------------
Number: 4.754
NumberFormat(4.754, 1, 2) = 4.75
Round2(4.754, 2) = 4.75
-------------------------------------------------------
Number: 4.764
NumberFormat(4.764, 1, 2) = 4.76
Round2(4.764, 2) = 4.76
-------------------------------------------------------
Number: 4.774
NumberFormat(4.774, 1, 2) = 4.77
Round2(4.774, 2) = 4.77
-------------------------------------------------------
Number: 4.784
NumberFormat(4.784, 1, 2) = 4.78
Round2(4.784, 2) = 4.78
-------------------------------------------------------
Number: 4.794
NumberFormat(4.794, 1, 2) = 4.79
Round2(4.794, 2) = 4.79
-------------------------------------------------------
Number: 4.705
NumberFormat(4.705, 1, 2) = 4.71
Round2(4.705, 2) = 4.71
-------------------------------------------------------
Number: 4.715
NumberFormat(4.715, 1, 2) = 4.71
Round2(4.715, 2) = 4.72
-------------------------------------------------------
Number: 4.725
NumberFormat(4.725, 1, 2) = 4.72
Round2(4.725, 2) = 4.72
-------------------------------------------------------
Number: 4.735
NumberFormat(4.735, 1, 2) = 4.74
Round2(4.735, 2) = 4.74
-------------------------------------------------------
Number: 4.745
NumberFormat(4.745, 1, 2) = 4.75
Round2(4.745, 2) = 4.75
-------------------------------------------------------
Number: 4.755
NumberFormat(4.755, 1, 2) = 4.75
Round2(4.755, 2) = 4.76
-------------------------------------------------------
Number: 4.765
NumberFormat(4.765, 1, 2) = 4.76
Round2(4.765, 2) = 4.76
-------------------------------------------------------
Number: 4.775
NumberFormat(4.775, 1, 2) = 4.78
Round2(4.775, 2) = 4.78
-------------------------------------------------------
Number: 4.785
NumberFormat(4.785, 1, 2) = 4.79
Round2(4.785, 2) = 4.79
-------------------------------------------------------
Number: 4.795
NumberFormat(4.795, 1, 2) = 4.79
Round2(4.795, 2) = 4.8
-------------------------------------------------------
Number: 4.706
NumberFormat(4.706, 1, 2) = 4.71
Round2(4.706, 2) = 4.71
-------------------------------------------------------
Number: 4.716
NumberFormat(4.716, 1, 2) = 4.72
Round2(4.716, 2) = 4.72
-------------------------------------------------------
Number: 4.726
NumberFormat(4.726, 1, 2) = 4.73
Round2(4.726, 2) = 4.73
-------------------------------------------------------
Number: 4.736
NumberFormat(4.736, 1, 2) = 4.74
Round2(4.736, 2) = 4.74
-------------------------------------------------------
Number: 4.746
NumberFormat(4.746, 1, 2) = 4.75
Round2(4.746, 2) = 4.75
-------------------------------------------------------
Number: 4.756
NumberFormat(4.756, 1, 2) = 4.76
Round2(4.756, 2) = 4.76
-------------------------------------------------------
Number: 4.766
NumberFormat(4.766, 1, 2) = 4.77
Round2(4.766, 2) = 4.77
-------------------------------------------------------
Number: 4.776
NumberFormat(4.776, 1, 2) = 4.78
Round2(4.776, 2) = 4.78
-------------------------------------------------------
Number: 4.786
NumberFormat(4.786, 1, 2) = 4.79
Round2(4.786, 2) = 4.79
-------------------------------------------------------
Number: 4.796
NumberFormat(4.796, 1, 2) = 4.8
Round2(4.796, 2) = 4.8
-------------------------------------------------------
Extra
-------------------------------------------------------
Number: 4.8
NumberFormat(4.8, 1, 2) = 4.8
NumberFormat2(4.8, 1, 2, 2, False) = 4.80
Round(4.8) = 5
Round2(4.8, 1) = 4.8
Round2(4.8, 2) = 4.8
-------------------------------------------------------
Number: 4.725
NumberFormat(4.725, 0, 2) = 4.72
NumberFormat(4.725, 1, 2) = 4.72
NumberFormat2(4.725, 0, 2, 2, False) = 4.72
NumberFormat2(4.725, 1, 2, 2, False) = 4.72
Round(4.725) = 5
Round2(4.725, 1) = 4.7
Round2(4.725, 2) = 4.72
Ceil(4.725) = 5
Floor(4.725) = 4
 

aeric

Well-Known Member
Licensed User

emexes

Well-Known Member
Licensed User
So, I think I will stick to NumberFormat2.
Excellent choice! I've not had any problems with NumberFormat2, and I use it a *lot*.

I did vaguely remember that it left the decimal point on even if there were no digits after the decimal point, but luckily I checked before mentioning it here, and either I've got it mixed up with another programming environment, or it's been fixed.
 

Mahares

Well Known Member
Licensed User
All functions result the same. So, I think I will stick to NumberFormat2.
Your answer in post #7 is: 4.72
But the answer you are looking for in post #1 is: 4.73. How did NumberFormat2 in post #7 is going to give you 4.73?
 

emexes

Well-Known Member
Licensed User
Your answer in post #7 is: 4.72
But the answer you are looking for in post #1 is: 4.73. How did NumberFormat2 in post #7 is going to give you 4.73?
Holy moley, you've got an eye for detail. I think this might be a case of Welcome to the Wonderful World of Floating Point Numbers.

The decimal fraction 4.725 is not exactly representable as a binary fraction, and is instead stored as the nearest available value which is apparently something along the lines of 4.72499999999999964472863211995 which rounds off to 4.72 (2 decimal places) or 4.725000000000000 (16 decimal digits of precision, which is about the limit of an IEEE double precision's 52-bit mantissa).

http://www.binaryconvert.com/result_double.html?decimal=052046055050053
 
Last edited:

emexes

Well-Known Member
Licensed User
if your value is money/currency double/float is not a good type to store this.
I 100% agree that float is no good, but I've found double is ok provided that you get your roundings right and that we're talking plausible numbers eg under a trillion dollars or whatever (so... not Zimbabwe ten years ago).

I'm not saying doubles are perfect, but I will say that currency data types have their own little traps for the unwary ;-)
 

aeric

Well-Known Member
Licensed User
Your answer in post #7 is: 4.72
But the answer you are looking for in post #1 is: 4.73. How did NumberFormat2 in post #7 is going to give you 4.73?
I will do experiment with different way. The input number may be look like 4.725 but in fact it is a floating point 4.72499999999999964472863211995 as mentioned by emexes above. This number may come from database query. So I may use SQL to round it first, cast it to other type or use report designer to format it. I'll see which way is the best. Maybe other data type like decimal, currency or money data type give different result.
 

emexes

Well-Known Member
Licensed User
I recall that we usually fixed the problem by using 0.501 as our rounding value, instead of the more-usual 0.5, eg:
B4X:
RoundedValue = Int(RawValue * 1000 + 0.501) / 1000
which works because the RawValue is clearly trying to be 4.725 precisely, but thanks to floating point vagaries it ends up being plus/minus a least-significant-bit eg 4.72500001 or 4.72499999.

If you round those numbers to 2 decimal places using 0.005 then you'll get 4.73 or 4.72, ie different results from nominally the same raw value = hmm.
If you round those numbers to 2 decimal places using 0.00501 then you'll get 4.73 and 4.73, which are consistent, and also the intended result, = yay!

You'd think that the slightly-higher rounding offset (501 cf 500) would cause problems to appear elsewhere, but in practice it doesn't, because the raw value is usually stepped in some way, eg if we were dealing with financial values then they would cluster around integer multiples of a cent.
 

emexes

Well-Known Member
Licensed User
The input number may be look like 4.725 but in fact it is a floating point 4.72499999999999964472863211995
I think your solution, given that it seems your input data was to 3 decimal places, might be to add a tiny bit eg 0.00001 which will convert:

- too-low-roundings eg 4.7249999junk to 4.7250099junk
- too-high-roundings eg 4.7250000junk to 4.7250100junk

and then you have the numbers with those 3 decimal places correct. The 4th decimal place will always be a zero (assuming that you're using doubles, and that the raw values are clustered around multiples of 0.001) and you can cut it and the following junk off.
 
Top