J JLS Member Licensed User Longtime User Nov 29, 2014 #1 Hi everybody: As my very first Project, I'm creating a calculator. I have found for example: 0.3 * 3 = 0.89999999999999 Why is this happening? How can it be fixed? Thanks in advance
Hi everybody: As my very first Project, I'm creating a calculator. I have found for example: 0.3 * 3 = 0.89999999999999 Why is this happening? How can it be fixed? Thanks in advance
S sorex Expert Licensed User Longtime User Nov 29, 2014 #2 it's because the way how computers work with floating numbers. you'll have to apply some rounding. Upvote 0
klaus Expert Licensed User Longtime User Nov 29, 2014 #3 To display numbers you should use NumberFormat or NumberFormat2. Upvote 0
J JLS Member Licensed User Longtime User Nov 29, 2014 #4 I suspected something like that. Thank you all for answer so fast. Upvote 0
K keirS Well-Known Member Licensed User Longtime User Nov 29, 2014 #5 klaus said: To display numbers you should use NumberFormat or NumberFormat2. Click to expand... To do calculations that require decimal precision you should not use floating point calculations. agrahm's BigNumbers library is the answer to this problem. It's not a display issue it's calculation issue. Upvote 0
klaus said: To display numbers you should use NumberFormat or NumberFormat2. Click to expand... To do calculations that require decimal precision you should not use floating point calculations. agrahm's BigNumbers library is the answer to this problem. It's not a display issue it's calculation issue.
klaus Expert Licensed User Longtime User Nov 29, 2014 #6 In most cases BigNumber calculations are not necessaray, Double calculations are enough. But the display is anoying, therefore the suggestion to use NumberFormat. Upvote 0
In most cases BigNumber calculations are not necessaray, Double calculations are enough. But the display is anoying, therefore the suggestion to use NumberFormat.