cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Floating point error problem in programming

SOLVED
Newbie

Floating point error problem in programming

I have done pressure calculations in normal worksheet format and i got results without any error. but it needs to change a value once for every iterations. So i moved on to programming. I compiled the program using while loop and break conditions. After completing and running the program i got error message showing "encountered a floating point error".Dont know how to fix the bug.. Can anyone help me pls?????????

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Floating point error problem in programming

Vigneshwar Venkatesan wrote:

I have done pressure calculations in normal worksheet format and i got results without any error. but it needs to change a value once for every iterations. So i moved on to programming. I compiled the program using while loop and break conditions. After completing and running the program i got error message showing "encountered a floating point error".Dont know how to fix the bug.. Can anyone help me pls?????????

Somewhere within your loop the deltaP .dc and .r variables are blowing up to the limits of numeric handling (ie, they're in the order of 10307 )

Stuart

View solution in original post

1 REPLY 1

Re: Floating point error problem in programming

Vigneshwar Venkatesan wrote:

I have done pressure calculations in normal worksheet format and i got results without any error. but it needs to change a value once for every iterations. So i moved on to programming. I compiled the program using while loop and break conditions. After completing and running the program i got error message showing "encountered a floating point error".Dont know how to fix the bug.. Can anyone help me pls?????????

Somewhere within your loop the deltaP .dc and .r variables are blowing up to the limits of numeric handling (ie, they're in the order of 10307 )

Stuart

View solution in original post

Announcements