Problem with Floats

[ondesic]

I have the following code

Dim testBool As Boolean
Dim testFloat As Float 

testFloat = 1.26

If testFloat < 1.26 Then 
    testBool = True
Else
    testBool = False
End If

testFloat ALWAYS solves as true. This is causing haywire problems in my code. testFloat and 1.26 should be equal, right?

I am using 2.52

 

[yttrium]

Does this happen when you try it with a Double instead of a Float?

 

[agraham]

testFloat and 1.26 should be equal, right?

No! You have encountered the normal effect of the in-exactitude of representing floating-point decimal numbers in binary form.

A Float has inherently less precision than a Double. The comparison is being made by converting the literal "1.26" directly to a Double value and comparing it to the Float converted to a Double. But the actual Float value when assigned "1.26" is something like 1.2599999904632568 or thereabouts. A Double can get much closer to 1.26 so the Float is less than the Double.

In general there is no reason to use Floats at all as Basic4android does all arithmetic with Double values.

 

[Erel]

In general there is no reason to use Floats at all as Basic4android does all arithmetic with Double values.

Small correction, this was true in Basic4ppc. Floats will not be casted to doubles if there is no need.

 

[ondesic]

I now understand the logic. However, can something be done to make this work like an average user would expect? This kicked my bum for a long time. Truthfully, I'm luck I narrowed it down to this. Just looking at the code for face value, an average user would think 1.26 should equal 1.26. Even in debug mode, there are no indications of this sneaky problem. When I hover over testFloat, it says "1.26".

 

[agraham]

This "problem" (it's not really a problem) is inherent in all languages and all architectures where decimal floating point values are represented in a binary format. Most programmers will find out about it sooner or later if they have benn not been taught about it on their Computer Science course. If you need absolute precision you need to use an appropriate type, like Int, Long or BigInteger for whole numbers or BigDecimals for floating point values.