Skip to main content
1-Visitor
November 8, 2012
Solved

Floating point error problem in programming

  • November 8, 2012
  • 1 reply
  • 1286 views

I have done pressure calculations in normal worksheet format and i got results without any error. but it needs to change a value once for every iterations. So i moved on to programming. I compiled the program using while loop and break conditions. After completing and running the program i got error message showing "encountered a floating point error".Dont know how to fix the bug.. Can anyone help me pls?????????

Best answer by StuartBruff

Vigneshwar Venkatesan wrote:

I have done pressure calculations in normal worksheet format and i got results without any error. but it needs to change a value once for every iterations. So i moved on to programming. I compiled the program using while loop and break conditions. After completing and running the program i got error message showing "encountered a floating point error".Dont know how to fix the bug.. Can anyone help me pls?????????

Somewhere within your loop the deltaP .dc and .r variables are blowing up to the limits of numeric handling (ie, they're in the order of 10307 )

Stuart

1 reply

23-Emerald V
November 8, 2012

Vigneshwar Venkatesan wrote:

I have done pressure calculations in normal worksheet format and i got results without any error. but it needs to change a value once for every iterations. So i moved on to programming. I compiled the program using while loop and break conditions. After completing and running the program i got error message showing "encountered a floating point error".Dont know how to fix the bug.. Can anyone help me pls?????????

Somewhere within your loop the deltaP .dc and .r variables are blowing up to the limits of numeric handling (ie, they're in the order of 10307 )

Stuart