Skip to main content
5-Regular Member
August 30, 2023
Solved

Rounding error with trunc() and floor() functions that return exact integers

  • August 30, 2023
  • 2 replies
  • 3121 views

I'm having an issue where I need to do division, then eliminate the decimal portion of the answer and only keep the integer. I've successfully used both trunc() and floor() for this application in the past. But when the quotient results in an exact integer (like 84/12=7 in my example), Mathcad rounds down to 6 instead of returning 7. I'm assuming this is the result of some approximation internal to how Mathcad does division. Like it's really calculating 6.99999999999999... maybe? Are there any work arounds for this? Obviously I can just make the equation 84/12, but this is in a template that I'm trying to make apply to all cases.

Best answer by Werner_E

Yes, the problem is because of numerical round offs - especially because of the usage of non SI standard unit (Prime stores values with SI units no matter how they are defined)

You may either add a small percentage to the nominator first

Werner_E_0-1693412047706.png

or get rid of the units by dividing b.road by the unit feet

Werner_E_1-1693412080748.png

 

2 replies

24-Ruby III
August 30, 2023

Hi,

it seems to me that the problem is related to ft unit somehow (I do not understand why).

In Mathcad Prime 9.0 following notation works well.

MartinHanak_1-1693411965546.png

 

 

24-Ruby III
August 31, 2023

MartinHanak_0-1693492827152.png

 

Werner_E25-Diamond IAnswer
25-Diamond I
August 30, 2023

Yes, the problem is because of numerical round offs - especially because of the usage of non SI standard unit (Prime stores values with SI units no matter how they are defined)

You may either add a small percentage to the nominator first

Werner_E_0-1693412047706.png

or get rid of the units by dividing b.road by the unit feet

Werner_E_1-1693412080748.png

 

21-Topaz II
August 30, 2023

Here is another way

Capture.JPG