Mathcad Prime 7.0.0.0 Floor and Trunc Errors for Integer Values with Units

[KM_10748216]

1. How can I correct the highlighted functions so that they return 1.00 ft (see image below)?

2. Are others experiencing similar issues in this version of Mathcad?

3. Is this issue resolved in later versions of the program?

Thanks!

[Werner_E · Accepted solution]

What you experience are (annoying) numerical inaccuracies because Prime and also real Mathcad will internally convert any unit to the corresponding basic SI unit.

Trunc and other functions of that kind suffer from numerical inaccuracies like

which are also shown as

Because of

when you are asking for 

Prime is actually calculating

and so runs into round-off errors even though you are providing and expecting only integer values (in feet).

Unfortunately Trunc, etc.  are resistant to the settings of the system variables CTOL and TOL.

A workaround could be the definition of your own truncation function

or

[KM_10748216]

Thanks for your insight. I will use the workaround you suggested.

[Werner_E]

Keep in mind that the workarounds may fail, too.

If, in the first suggestion, a/b should be 1 but because of round-off errors happen to be 0.99999999995675, the function will fail, too, because the floor() function will return 0 instead of 1).