cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Community Tip - You can Bookmark boards, posts or articles that you'd like to access again easily! X

Calculating error?

ptc-1215941
1-Newbie
1-Newbie
‎Dec 07, 2009 04:04 PM
‎Dec 07, 2009 04:04 PM

Calculating error?

When I caculate 107 * 1.04 with 17 decimal places I get 112.35000000000001000 instead of 112.35000000000000000. See attached MCAD 14 file Does anyone else get this result?
Labels:
problem_151.xmcd
0 Kudos
Notify Moderator
1 REPLY 1
RichardJ
19-Tanzanite
19-Tanzanite
(To:ptc-1215941)
‎Dec 07, 2009 04:29 PM
‎Dec 07, 2009 04:29 PM

Welcome to the world of numerical roundoff. You only get about 16 digits of precision. Richard
0 Kudos
Notify Moderator
Top Tags