cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Did you know you can set a signature that will be added to all your posts? Set it here! X

Translate the entire conversation x

Error in logical statements (Floating points error?)

BAustara
2-Explorer

Error in logical statements (Floating points error?)

There seems to be an error regarding logical statements in Prime 10.0.1. See below.
The expected output of all statements is 1. It seems to only be a problem when using units.

BA_13569641_0-1748415581253.png

 Is there a fix to this? Thanks

ACCEPTED SOLUTION

Accepted Solutions
Werner_E
25-Diamond I
(To:BAustara)

What you experience is a combination of inaccuracies in unit conversion (even though its only a multiplication by 10^-3 as Prime is always going back to the base units) and the 'natural' accuracy limit of the IEEEE number format used.

A 'fix" is to use a system which is using a number format which can provide more accuracy then the typical 14-16 decimals of the IEEE storage format.

Primes built-in symbolic engine provides more accuracy and could be used. Using the symbolic evaluation (->) instead of the numeric (=) one gives the correct result for your example. You may want to check "Unit/Constants in Symbolics" in the Calculation Options.

Werner_E_0-1748437116614.png

 

Sometimes it may be necessary to add the "simplify" modifier to get a meaningful result:

Werner_E_1-1748437273738.png

 

 

View solution in original post

1 REPLY 1
Werner_E
25-Diamond I
(To:BAustara)

What you experience is a combination of inaccuracies in unit conversion (even though its only a multiplication by 10^-3 as Prime is always going back to the base units) and the 'natural' accuracy limit of the IEEEE number format used.

A 'fix" is to use a system which is using a number format which can provide more accuracy then the typical 14-16 decimals of the IEEE storage format.

Primes built-in symbolic engine provides more accuracy and could be used. Using the symbolic evaluation (->) instead of the numeric (=) one gives the correct result for your example. You may want to check "Unit/Constants in Symbolics" in the Calculation Options.

Werner_E_0-1748437116614.png

 

Sometimes it may be necessary to add the "simplify" modifier to get a meaningful result:

Werner_E_1-1748437273738.png

 

 

Announcements


Top Tags