B4R Question 8 Byte Double?

Computersmith64

Well-Known Member
Licensed User
Longtime User
Is the 4 byte double a limitation of the ESP8266 boards, or is it just that B4R doesn't have an 8 byte floating point data type?

I'm working on a project where I have to monitor large quantities of liquids (10,000 litres +) & have to have resolution down to the millilitre level, however I'm finding that with the 4 byte double (or float) type available in B4R, I'm losing precision when I have to subtract millilitre quantities from large values. The result is that my increasing "used" counter gets out of sync with the decreasing "level" indicator. I don't see this issue with quantities less than 10,000 litres.

- Colin.
 

JordiCP

Expert
Licensed User
Longtime User
This doesn't directly answer your question , but you can work internally only with integers representing millilitres (or the resolution you need). That is, everything multiplied by 1000. You will get rid of problems with doubles/floats precision.
 
Upvote 0

Computersmith64

Well-Known Member
Licensed User
Longtime User
This doesn't directly answer your question , but you can work internally only with integers representing millilitres (or the resolution you need). That is, everything multiplied by 1000. You will get rid of problems with doubles/floats precision.
I did try that, but because I'm using tenths & hundredths of ml I still had the same issue. Thinking about it, I might be able to multiply the ml by 100 & the litres by 100,000 to get the precision I need. Values up to about 40,000 litres should still fit into a 4 byte double when multiplied by 100,000. (I think)

- Colin.
 
Upvote 0

Computersmith64

Well-Known Member
Licensed User
Longtime User
I did try that, but because I'm using tenths & hundredths of ml I still had the same issue. Thinking about it, I might be able to multiply the ml by 100 & the litres by 100,000 to get the precision I need. Values up to about 40,000 litres should still fit into a 4 byte double when multiplied by 100,000. (I think)

- Colin.
Nope - tried that too, but I'm still losing precision (I guess due to rounding during the multiplication & then division) & my counters are still getting out of sync.

- Colin.
 
Upvote 0
Top