Please post the Mathcad file containing your equations that give the error messages. Mathcad shouldn't have numerical difficulties with a problem like this. Maybe some kind of simplification is needed?
Another solution is to use a version of the law of sines with the diagonal line drawn in, with the constraint that the sum of the angles between the lower side triangle and the diagonal line triangle sum to 45 degrees. Using crude initial guess values with a Given statement and the minerr function gives a solution.