cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Large Numbers Round On Their Own In Parameters

SOLVED
Highlighted
Level 7

Large Numbers Round On Their Own In Parameters

When I input a number that is greater than 16 digits, it begins to round. For example, I insert the following:

2222222222222222222222222.00000

and it automatically corrects it to:

2222222222222222184939520.000000

 

I don't understand why it is doing this. I need these numbers to be exact becuase it is a code that drives parameters based on the numbers 3 and 2. 

 

For instance, if I put 232323223, an interchange will grab the part associated with each individual number. It is not being using for a dimensional value.

 

Thanks,

 John

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Large Numbers Round On Their Own In Parameters

In the end, I used the Extract function.

 

IF EXTRACT(CODEBASESTRING,1,1)== "3"
D10=30
BRIDGE="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,1,1) == "2"
D10=20
BRIDGE="BRIDGE1"
ELSE
BRIDGE="NA"
D10=0
ENDIF
ENDIF

 


IF EXTRACT(CODEBASESTRING,2,1) == "3"
D40=30
BRIDGE2="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,2,1) == "2"
D40=20
BRIDGE2="BRIDGE1"
ELSE
BRIDGE2="NA"
D40=0
ENDIF
ENDIF

 

Though I get an "Probable Error" for the digits that do not exist (because the CODEBASESTRING is too short). I would suggest only using this if an empty cell is used for something not existing. For example, I have 75 total pieces. If I only have a code that is STRING_LENGTH(CODEBASESTRING) =  7, then the pieces past 7 are not included in the assembly. You should do the same for the code as well.

 

Cheers,

   Jonathyn

9 REPLIES 9

Re: Large Numbers Round On Their Own In Parameters

Can you use a string parameter to drive your interchange instead of a floating number? That would make it cleaner and there would be no rounding. Possibly an integer may be an alterntive.

Re: Large Numbers Round On Their Own In Parameters


@pmajor wrote:

When I input a number that is greater than 16 digits, it begins to round. For example, I insert the following:

2222222222222222222222222.00000

and it automatically corrects it to:

2222222222222222184939520.000000

 

I don't understand why it is doing this. I need these numbers to be exact becuase it is a code that drives parameters based on the numbers 3 and 2. 

 

For instance, if I put 232323223, an interchange will grab the part associated with each individual number. It is not being using for a dimensional value.

 

Thanks,

 John

 


I would suggest using a string parameter for this since it is actually a part number and not a numeric value.

Re: Large Numbers Round On Their Own In Parameters

The code is configured numerically. For instance, 23232223. I take the mod,10 of this number to recieve the last digit. Floor,10 to get the next number. It needs to be a real number. I am assuming from your answer that there isn't a way to achieve this?

 

 

Re: Large Numbers Round On Their Own In Parameters


@pmajor wrote:

The code is configured numerically. For instance, 23232223. I take the mod,10 of this number to recieve the last digit. Floor,10 to get the next number. It needs to be a real number. I am assuming from your answer that there isn't a way to achieve this?

 


Real numbers will always have this issue:

http://analytics.ncsu.edu/sesug/2008/PO-082.pdf

http://www.lexjansen.com/phuse/2011/pp/PP12.pdf

http://support.sas.com/resources/papers/proceedings11/275-2011.pdf

 

You could probably use an integer parameter to do what you are trying to do. However it sounds like what you are attempting to do is extract each individual digit. This is what strings are good at. If you can then post the complete code on how you are actually extracting these digits.

Re: Large Numbers Round On Their Own In Parameters


@BenLoosli wrote:

Can you use a string parameter to drive your interchange instead of a floating number? That would make it cleaner and there would be no rounding. Possibly an integer may be an alterntive.


Geez I completely missed this reply...

You must have been typing faster than me.

Re: Large Numbers Round On Their Own In Parameters

I will try the solution 

 

Re: Large Numbers Round On Their Own In Parameters

I think I misplaced an ")" snuck in my example. I'll edit the related post.

Re: Large Numbers Round On Their Own In Parameters

In the end, I used the Extract function.

 

IF EXTRACT(CODEBASESTRING,1,1)== "3"
D10=30
BRIDGE="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,1,1) == "2"
D10=20
BRIDGE="BRIDGE1"
ELSE
BRIDGE="NA"
D10=0
ENDIF
ENDIF

 


IF EXTRACT(CODEBASESTRING,2,1) == "3"
D40=30
BRIDGE2="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,2,1) == "2"
D40=20
BRIDGE2="BRIDGE1"
ELSE
BRIDGE2="NA"
D40=0
ENDIF
ENDIF

 

Though I get an "Probable Error" for the digits that do not exist (because the CODEBASESTRING is too short). I would suggest only using this if an empty cell is used for something not existing. For example, I have 75 total pieces. If I only have a code that is STRING_LENGTH(CODEBASESTRING) =  7, then the pieces past 7 are not included in the assembly. You should do the same for the code as well.

 

Cheers,

   Jonathyn

Re: Large Numbers Round On Their Own In Parameters

This is disturbing - my earlier post is gone. Lithium problem?

 

It included a link to https://en.wikipedia.org/wiki/Double-precision_floating-point_format which pointed out that the IEEE Double precision format is limited to between 15 and 17 decimal digits because it used 53 bits for the significant value. 53* log(10)/log(2) ~= 15.9

 

It also included the following relations:

(rel_model_name is a string parameter for which each character is extracted and then used to reconstruct a value)

 

/*Without comments 
digitlist = "0123456789" 
value = 0 
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-0,1))-1 
value = value+digit_found 
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-1,1))-1 
value = value+digit_found*10 
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-2,1))-1 
value = value+digit_found*100 
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-3,1))-1 
value = value+digit_found*1000 
d0=value