Skip to main content
1-Visitor
December 14, 2015
Question

Getting different results for the same argument? Why

  • December 14, 2015
  • 4 replies
  • 2858 views

Hi All,

Could you please have a look at the attached file and let me know why I am getting different answers?

if the number in question is an input you I get the answer I am looking for, if the output is a result form say interpolation I will get wrong answer!!!

Why, this is happening, and how to overcome it?

Regards,

H

4 replies

25-Diamond I
December 14, 2015

This is an effect which you have to take in account when you are working with float numbers. Never compare float numbers for equality.

Its a roundoff effect. While the interpolated value R should be exactly 17.85 its actually slightly smaller - approx. -3.5*10^(-15). Evaluate R-Rd to see it yourself.

WE

h_nas1-VisitorAuthor
1-Visitor
December 14, 2015

Hellow,

How can I overcome this issue then? Any advise

19-Tanzanite
December 14, 2015

On the "Calculation" tab, select "Calculation Options" and "Approximate Equality".