cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Need help navigating or using the PTC Community? Contact the community team. X

Extra digits?

IRstuff
3-Visitor

Extra digits?

MP1 shows up with 2 extra digits of precision over M11, but where did that come from? Did they change the floating point format?

Also, pi and e appear to be off by 1 in the last digit.

TTFN,
Eden
4 REPLIES 4

Eden,

The floating point did not change, but I can check on the details. The last digit difference is from rounding. This came up in the Alpha test.

Mona
IRstuff
3-Visitor
(To:mzeftel)

Actually, there's an error in the display routine, between full precision decimal and scientific.

This is asin(0.5) with the different display modes

(* 5.2359877559829888 (^ 10 (@NEG 1)))
0.52359877559829893

Depending on which display mode is called out, the displayed precision is different, and, in the case of fixed point, erroneously displayed

the "correct" value from M11 symbolic engine:
.52359877559829887308

TTFN,
Eden

Eden,

I logged it.

Mona

It will be very very good to format not digits number after decimal point but common digits number in the numerical answer. See our discuss on this subject.

One engineering task -
http://twt.mpei.ac.ru/MCS/Worksheets/WSP/VPT-SI.xmcd
Val
http://twt.mpei.ac.ru/ochkov/v_ochkov.htm
Top Tags