Skip to main content
Best answer by RichardJ

Roundoff.

Got to be careful making comparisons of floating point numbers when your worksheet options are set to "Use exact equality for comparisons and truncation"

3 replies

RichardJ19-TanzaniteAnswer
19-Tanzanite
August 21, 2012

Roundoff.

Got to be careful making comparisons of floating point numbers when your worksheet options are set to "Use exact equality for comparisons and truncation"

ValeryOchkov24-Ruby IVAuthor
24-Ruby IV
August 22, 2012

Richard Jackson wrote:

Roundoff.

Got to be careful making comparisons of floating point numbers when your worksheet options are set to "Use exact equality for comparisons and truncation"

Thanks Richard, I knew it.

But the paradox is a paradox.

Life (Mathcad) would be uninteresting without paradoxes.

RichardJ19-Tanzanite
19-Tanzanite
August 22, 2012

I think I recall something like this coming up on the forums once before. Mathcad can be a little deceptive when using units, because although you may enter the numerical part of a quantity as "1" (e.g. 1 lb), which has an exact binary representation, it is not necessarily stored internally as "1".

Terms & ConditionsAccessibility statement