Skip to main content
K
KM_107482161-Visitor
1-Visitor
July 31, 2023
Solved

Mathcad Prime 7.0.0.0 Floor and Trunc Errors for Integer Values with Units

  • July 31, 2023
  • 1 reply
  • 1224 views

1. How can I correct the highlighted functions so that they return 1.00 ft (see image below)?

2. Are others experiencing similar issues in this version of Mathcad?

3. Is this issue resolved in later versions of the program?

Thanks!

 

KM_10748216_0-1690816991945.png

 

Best answer by Werner_E

What you experience are (annoying) numerical inaccuracies because Prime and also real Mathcad will internally convert any unit to the corresponding basic SI unit.

 

Trunc and other functions of that kind suffer from numerical inaccuracies like

Werner_E_0-1690821445212.png

which are also shown as

Werner_E_1-1690821476269.png

 

Because of

Werner_E_4-1690821567198.png

when you are asking for 

Werner_E_8-1690822094937.png
Prime is actually calculating

Werner_E_3-1690821539389.png

and so runs into round-off errors even though you are providing and expecting only integer values (in feet).

 

Unfortunately Trunc, etc.  are resistant to the settings of the system variables CTOL and TOL.

 

A workaround could be the definition of your own truncation function

Werner_E_6-1690821954802.png

or

Werner_E_7-1690821967458.png

 

 

1 reply

W
Werner_E25-Diamond IAnswer
25-Diamond I
July 31, 2023

What you experience are (annoying) numerical inaccuracies because Prime and also real Mathcad will internally convert any unit to the corresponding basic SI unit.

 

Trunc and other functions of that kind suffer from numerical inaccuracies like

Werner_E_0-1690821445212.png

which are also shown as

Werner_E_1-1690821476269.png

 

Because of

Werner_E_4-1690821567198.png

when you are asking for 

Werner_E_8-1690822094937.png
Prime is actually calculating

Werner_E_3-1690821539389.png

and so runs into round-off errors even though you are providing and expecting only integer values (in feet).

 

Unfortunately Trunc, etc.  are resistant to the settings of the system variables CTOL and TOL.

 

A workaround could be the definition of your own truncation function

Werner_E_6-1690821954802.png

or

Werner_E_7-1690821967458.png

 

 

K
KM_107482161-VisitorAuthor
1-Visitor
July 31, 2023

Thanks for your insight. I will use the workaround you suggested.

W
Werner_E25-Diamond I
25-Diamond I
July 31, 2023

Keep in mind that the workarounds may fail, too.

If, in the first suggestion, a/b should be 1 but because of round-off errors happen to be 0.99999999995675, the function will fail, too, because the floor() function will return 0 instead of 1).

Terms & ConditionsCookie settingsAccessibility statement