[Bug?] Float to Label.Text

Discorez · Aug 15, 2012

I have a rounded float variable (in debugger I see it as 7.908), and I show
result in two labels, but I receive different results:

B4X:

Sub btCalc_Click
   Dim Rez As Float
   
   Rez = Round2(7.907999456789, 3) ' Rez debugger shows as "7.908"
   lbResult2.Text = Rez                    ' shows as "7.908"
        lbResult.Text = Rez & " mV"          ' shows as "7.907999456789 mV"
End Sub

:BangHead: Why?

test code is attached

Upd.

B4X:

lbResult.Text = NumberFormat(Rez, 0, 3) & " mV"

It works correctly, but the situation which I described above is unclear for me...

klaus · Aug 15, 2012

If you look at the help Round2 returns a Double.
But you define a Float in the routine.
This works:

B4X:

Sub btCalc_Click
    Dim Rez As Double
    
    Rez = Round2(7.907999456789,3)
    lbResult.Text = Rez & " mV"
    lbResul2.Text = Rez
End Sub

But I agree it looks a bit strange.

Best regards.

Erel · Aug 16, 2012

The best way to format a number is with NumberFormat(2) keywords.

What you are seeing is a result of a rounding error (because you used Float instead of Double). In the first case the number is first converted to double and in the second case it doesn't.

Discorez · Aug 16, 2012

Erel, Claus, thanks.
I understood this.
But why debugger shows value for Rez variable as 7.908 instead 7.907999456789?
It should be?

Erel · Aug 17, 2012

This is related to the way that the number is converted to string by the debugger.

[Bug?] Float to Label.Text

Discorez

Member

Attachments

klaus

Expert

Erel

B4X founder

Discorez

Member

Erel

B4X founder

Similar Threads

[Bug?] Float to Label.Text

Discorez

Member

Attachments

klaus

Expert

Erel

B4X founder

Discorez

Member

Erel

B4X founder

Similar Threads

Privacy & Transparency

Privacy & Transparency