cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - You can Bookmark boards, posts or articles that you'd like to access again easily! X

Getting different results for the same argument? Why

h_nas
4-Participant

Getting different results for the same argument? Why

Hi All,

Could you please have a look at the attached file and let me know why I am getting different answers?

if the number in question is an input you I get the answer I am looking for, if the output is a result form say interpolation I will get wrong answer!!!

Why, this is happening, and how to overcome it?

Regards,

H

4 REPLIES 4
Werner_E
25-Diamond I
(To:h_nas)

This is an effect which you have to take in account when you are working with float numbers. Never compare float numbers for equality.

Its a roundoff effect. While the interpolated value R should be exactly 17.85 its actually slightly smaller - approx. -3.5*10^(-15). Evaluate R-Rd to see it yourself.

WE

h_nas
4-Participant
(To:Werner_E)

Hellow,

How can I overcome this issue then? Any advise

RichardJ
19-Tanzanite
(To:h_nas)

On the "Calculation" tab, select "Calculation Options" and "Approximate Equality".

h_nas
4-Participant
(To:RichardJ)

Thank you

Announcements

Top Tags