Variable error?

Pantelis

Member
Licensed User
Longtime User
Hi

I am trying to just set a numeric value to a variable and then read the variable but i get strange results

B4X:
Dim A As Float
Dim B As Float
   
A=3.44
Msgbox(A,"A") 'gives 3.440000057220459
   
B = Round2(A,2) 
Msgbox(B,"B")   'gives again 3.440000057220459

Why i get that results from Variable A? I don't do any calculation there, i just set the value to 3.44. Why i get 3.440000057220459?

Then i Round the value of variable A with 2 fractional digits only and set it to variable B. How is it possible to get that result again? Round2 function doesn't work?
 
Last edited:

Pantelis

Member
Licensed User
Longtime User

i wnow about the floating-point effect.

In visual basic gives me results like this after a few calculations. But here i just set a value. The above example gives me 3.44 in variable A in visual basic

And how about round? How to show to the user 3.44 if round doesn't work? In visual basic if after some calculations that affect appears then i used a round and get normal results again.
 
Last edited:
Upvote 0

Pantelis

Member
Licensed User
Longtime User
Note that the correct way to convert a number to string is with NumberFormat or NumberFormat2 keywords.

Hi Erel. Probably you refer on how to show the result to the user with the right fractional digits. But what if i want to search the value on a table? How to round it to prefered digits and then to lookup the table?
 
Upvote 0

Pantelis

Member
Licensed User
Longtime User
Correction.
Visual basic gives me 3.44 if i use

B4X:
Label1.text = A

If i use
B4X:
 Label1.Text = Math.Round(a, 15)

then i have the same effect "3.44000005722046"
 
Last edited:
Upvote 0

Pantelis

Member
Licensed User
Longtime User
BTW, if you want to search for the value then you can convert it to string with NumberFormat and then search for the string value.

Thanks for the idea Erel.
What if my table has float number? How to check the equality?
 
Last edited:
Upvote 0

mc73

Well-Known Member
Licensed User
Longtime User
What is the precision you are after? You can multiply by a n-th power of 10, round to 0 decimals and compare.
 
Upvote 0

Pantelis

Member
Licensed User
Longtime User
Thanks for the reply mc73.
I want the accuracy of two fractional digits. I tried the method you suggested but it didn't gave me always correct results. For example.


B4X:
Dim a As Float
Dim b As Float

 a = 0
 For i = 0 To 29
    a = a + 0.1
 Next 
 a = a + 0.445             'The result must be 3.445
 Msgbox (a,"a")          'But it is 3,444992179870605
 
 b = 6
 For i = 0 To 29
    b = b - 0.1
 Next
 b = b + 0.445      'The result must be 3.445
 Msgbox (b,"b")       'but it is 3,445002794265747

Msgbox(Round2((a * 100),0),"Check A")   'The result is 344
Msgbox(Round2((b * 100),0),"Check B")   'The result is 345
 
If Round2((a * 100),0) = Round2((b * 100),0) Then
   Msgbox ("equal","Result")
Else
   Msgbox ("Not equal","Result")
End If

After some calculations, variable “a” must have 3,445 value but the real value is 3,444992179870605.
Variable “b” after some other calculations must be 3,445 also but it is 3,445002794265747

So in “IF” statement, variable “a” has the value of 344 and variable “b” the value of 345 and of course the returned value is “Not equal”.
 
Upvote 0

Sonny

Member
Licensed User
Longtime User
Erel's right; if you change the declarations from "float" to "double" your code will now test "true". The "Round2" function you're calling requires a double in order to function properly. Passing it a float will only create problems as the remaining, unused, normally significant decimal places, are in an uninitialized state, and can contain any random value to throw off your logic.

> Sonny <
 
Last edited:
Upvote 0

Pantelis

Member
Licensed User
Longtime User
Hi Sonny.

Try this code:

B4X:
Dim a As Double
Dim b As Double

 a = 30.005
 For i = 0 To 29
     a = a + 0.1
 Next 
 
 Msgbox (a,"a")            
 
 b = 36.005
 For i = 0 To 29
     b = b - 0.1
 Next
 
 Msgbox (b,"b")       

Msgbox(Round2((a * 100),0),"Check A")    
Msgbox(Round2((b * 100),0),"Check B")    
 
If Round2((a * 100),0) = Round2((b * 100),0) Then
    Msgbox ("equal","Result")
Else
    Msgbox ("Not equal","Result")
End If

Again "Not equal".
 
Upvote 0

mc73

Well-Known Member
Licensed User
Longtime User
I was not clear enough, cause I didn't have source code available. You should multiply by the desired factor, ALL values. This way you will deal with integers. I am not sure if this a good solution, but works for me, so you can give it a try, if nothing more suitable appears.

Example:
B4X:
Sub calculation
    Dim a As Double
    Dim b As Double
    a = fc(30.005)
    For i = 0 To 29
        a = a + fc(0.1)
    Next 
    a=dfc(a)
    Msgbox (a,"a")            
     
    b = fc(36.005)
    For i = 0 To 29
        b = b - fc(0.1)
    Next
    b=dfc(b)
    Msgbox (b,"b")       

    If a = b Then
        Msgbox ("equal","Result")
    Else
        Msgbox ("Not equal","Result")
    End If
End Sub

Sub fc(yourNumber As Double) As Int
    Dim factor As Int = 100
    Return Round2(yourNumber*factor,0)
End Sub

Sub dfc(yourNumber As Int) As Double 
    Return Round2(yournumber/100,2)
End Sub
 
Upvote 0

Sonny

Member
Licensed User
Longtime User
Pantelis,

Dude, you're cheating.

You're multiplying the results by 100 BEFORE you perform the Round2 function. As a result, the decimal portion - that would normally be "adjusted" by the Round2 call, so that you would get a TRUE result - is now in the integer portion of your number, and as thus, has "artificially" become significant.

In your original example, the numbers you chose to work with did not have decimal portions of a magnitude high enough to make a difference. In your original example, the results were; [A = 3.445000000000001] and [B = 3.445000000000006], so the effect of multiplying them both by 100 is irrelevant. In your second example, the results were; [A = 33.00500000000004] and [B = 33.00499999999996]. So now, multiplying them both by 100 yields the rounding error you see. You also left out the original "correction factor" that you applied to both A & B in the first example of; "a = a + 0.445" and "b = b + 0.445". If you left these in place in your second example, then the rounding would have yielded a TRUE result.

In doing such "out of sequence manipulations", you have violated the rules of significant figures - you've introduced greater accuracy into your results than your measuring equipment is capable of producing.

This is why there is a specific order of operations that must be respected, otherwise you'll get incorrect results.

> Sonny <
 
Last edited:
Upvote 0

Pantelis

Member
Licensed User
Longtime User
Sonny,

I am multiplying by 100 before Round2, as suggested by mc73, because if i use only round2 for rounding and then do the checking of the equality of float or double variables i don't have the needed accuracy. Again with some numbers returning "not equal"

I changed the number for my second example and removed the "correction factor" trying to examine if setting variable types to double gave me accuracy in equality checking with every possible number.

What method do you suggest?
 
Upvote 0

Sonny

Member
Licensed User
Longtime User
Sonny,

I am multiplying by 100 before Round2, as suggested by mc73, because if i use only round2 for rounding and then do the checking of the equality of float or double variables i don't have the needed accuracy. Again with some numbers returning "not equal"

I changed the number for my second example and removed the "correction factor" trying to examine if setting variable types to double gave me accuracy in equality checking with every possible number.

What method do you suggest?


Pantelis,

When you say "accuracy", what is it are you really after? Accuracy as in 2 decimal places, ie; 100.23 or 92.71? Please explain.

Here's an example of what we used to do back in the "wonderful days of Fortran programming" a long time ago (back when I still had hair). In this example, we're looking for 2 decimal places of rounding and "accuracy":

- assume that your result is; 100.234xxxxxxxxx
- and you want a rounded accuracy of; 100.xx
- so add "0.005" to the result -> you get; 100.239xxxxxxxxx
- multiply the result by 100 -> you get; 10023.9xxxxxxxxx
- take the INT of the number -> you get; 10023
- divide the number by 100 -> you get; 100.23

When you add 0.005 to the number, this causes 0.234 to advance to 0.239 which will give you the correct rounding when you perform the other steps.

If the number were 100.235 instead, then the result would be; 100.240; so when you perform the remaining steps you'll get; 100.24; which is what you're after, right?

This is what the Round2 function does - behind the scenes - so if you multiply before you Round2, you'll lose the desired effect. Round2 first, then multiply.

Round2(Number as double, Decimal Places as int) as double

"result = Round(100.234xxxxxxxxx, 2)" should return the same result.

You were doing; "Round2((a * 100),0)". The zero means "don't perform any decimal rounding at all - just truncate the entire decimal portion and return the integer portion."

Read the documentation - it states; "Rounds the given number and leaves up to the specified number of fractional digits", meaning, if you want to round to a precision of; "abc.xy", then replace the zero with "2".

I hope I'm understanding your question correctly, and am giving you useful information.

> Sonny <
 
Upvote 0
Top