Skip to main content
R
rspears1-Visitor
1-Visitor
June 2, 2015
Question

Multiplying a variable by "ft" changes the value - MathCAD bug

  • June 2, 2015
  • 4 replies
  • 2584 views

I am having an issue with variables changing value from an integer to an integer +/- a very small value like 0.0000000000000002 when I assign the variable a unit. I've changed the default units to U.S. thinking that there was some small conversion being done incorrectly, but the issue is still there. Does anyone have any idea what could be causing this? This is really putting a kink in my work day. I have to subtract these values from other values and then test that result. If that result is < 0 instead of exactly 0, the MathCAD sheet doesn't work correctly. To me, this is a major issue, and I'm not sure why MathCAD would have a bug like this. I've included a screenshot for clarity on what I'm talking about.

mathcadissue.PNG

4 replies

R
RichardJ19-Tanzanite
19-Tanzanite
June 2, 2015

It's not a bug. It's inherent to numerical computing. All numbers in Mathcad are stored as double precision floats, which gives you about 15-16 digits of precision. What you are looking at are numerical round-off errors, and it's why you should never do an exact comparison of two floating point numbers (a rule that is universal, not just for Mathcad). Go to "Tools", "Worksheet options", and select the "Calculation" tab. Uncheck "Use exact equality for comparisons and truncation". Unfortunately, the default is for this to be checked, which may not be a bug, but it's a very bad choice for the default.

R
rspears1-VisitorAuthor
1-Visitor
June 2, 2015

I unchecked the box you described and it doesn't solve my problem. The spreadsheet still says that 0.00000000000000002 is greater than 0 and my sheet still doesn't get the correct answer. I tried rounding the value that is being compared to 0 and apparently you can't round a value with units.

R
RichardJ19-Tanzanite
19-Tanzanite
June 2, 2015

Please post the worksheet (Click on "Use advanced editor" at the top right of the edit dialog").

Terms & ConditionsAccessibility statement