For some reason the picture you embedded is not visible.
Obviously you are using the the symbolic "solve" command to solve an equation an get numeric results. Results of symbolic calculations cannot be formatted via the math format tab.
You can add something like "float,5" as a modifier to the symbolic command to limit the calculation to five (or any other number of) significant digits. But generally its a bad idea to do so as this limit affects the whole calculation, not just the display of the result.
You may consider assigning the result of the symbolic solve to a variable and then display this variable using the normal math formatting.
As you are just after numerical results you may also consider using a solve block or the "root" function instead of the symbolic solve. But both will just return one solution depending on a guess value you have to provide.
For a simple quadratic equation you may also use the "polyroot" function to get both solutions.
BTW, its always better to additionally attach our worksheet and not just post pictures.
Are you really Using Prime 3.1? Its rather old and outdated now.
