Large Numbers Round On Their Own In Parameters
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Large Numbers Round On Their Own In Parameters
When I input a number that is greater than 16 digits, it begins to round. For example, I insert the following:
2222222222222222222222222.00000
and it automatically corrects it to:
2222222222222222184939520.000000
I don't understand why it is doing this. I need these numbers to be exact becuase it is a code that drives parameters based on the numbers 3 and 2.
For instance, if I put 232323223, an interchange will grab the part associated with each individual number. It is not being using for a dimensional value.
Thanks,
John
Solved! Go to Solution.
- Labels:
-
General
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
In the end, I used the Extract function.
IF EXTRACT(CODEBASESTRING,1,1)== "3"
D10=30
BRIDGE="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,1,1) == "2"
D10=20
BRIDGE="BRIDGE1"
ELSE
BRIDGE="NA"
D10=0
ENDIF
ENDIF
IF EXTRACT(CODEBASESTRING,2,1) == "3"
D40=30
BRIDGE2="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,2,1) == "2"
D40=20
BRIDGE2="BRIDGE1"
ELSE
BRIDGE2="NA"
D40=0
ENDIF
ENDIF
Though I get an "Probable Error" for the digits that do not exist (because the CODEBASESTRING is too short). I would suggest only using this if an empty cell is used for something not existing. For example, I have 75 total pieces. If I only have a code that is STRING_LENGTH(CODEBASESTRING) = 7, then the pieces past 7 are not included in the assembly. You should do the same for the code as well.
Cheers,
Jonathyn
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Can you use a string parameter to drive your interchange instead of a floating number? That would make it cleaner and there would be no rounding. Possibly an integer may be an alterntive.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
The code is configured numerically. For instance, 23232223. I take the mod,10 of this number to recieve the last digit. Floor,10 to get the next number. It needs to be a real number. I am assuming from your answer that there isn't a way to achieve this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
@pmajor wrote:
The code is configured numerically. For instance, 23232223. I take the mod,10 of this number to recieve the last digit. Floor,10 to get the next number. It needs to be a real number. I am assuming from your answer that there isn't a way to achieve this?
Real numbers will always have this issue:
http://analytics.ncsu.edu/sesug/2008/PO-082.pdf
http://www.lexjansen.com/phuse/2011/pp/PP12.pdf
http://support.sas.com/resources/papers/proceedings11/275-2011.pdf
You could probably use an integer parameter to do what you are trying to do. However it sounds like what you are attempting to do is extract each individual digit. This is what strings are good at. If you can then post the complete code on how you are actually extracting these digits.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I will try the solution dschenken has provided. I will post code if I can get it to work.
Cheers
-Jon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I think I misplaced an ")" snuck in my example. I'll edit the related post.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
This is disturbing - my earlier post is gone. Lithium problem?
It included a link to https://en.wikipedia.org/wiki/Double-precision_floating-point_format which pointed out that the IEEE Double precision format is limited to between 15 and 17 decimal digits because it used 53 bits for the significant value. 53* log(10)/log(2) ~= 15.9
It also included the following relations:
(rel_model_name is a string parameter for which each character is extracted and then used to reconstruct a value)
/*Without comments
digitlist = "0123456789"
value = 0
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-0,1))-1
value = value+digit_found
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-1,1))-1
value = value+digit_found*10
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-2,1))-1
value = value+digit_found*100
digit_found = search(digitlist, extract(rel_model_name, len(rel_model_name)-3,1))-1
value = value+digit_found*1000
d0=value
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
In the end, I used the Extract function.
IF EXTRACT(CODEBASESTRING,1,1)== "3"
D10=30
BRIDGE="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,1,1) == "2"
D10=20
BRIDGE="BRIDGE1"
ELSE
BRIDGE="NA"
D10=0
ENDIF
ENDIF
IF EXTRACT(CODEBASESTRING,2,1) == "3"
D40=30
BRIDGE2="BRIDGE2"
ELSE
IF EXTRACT(CODEBASESTRING,2,1) == "2"
D40=20
BRIDGE2="BRIDGE1"
ELSE
BRIDGE2="NA"
D40=0
ENDIF
ENDIF
Though I get an "Probable Error" for the digits that do not exist (because the CODEBASESTRING is too short). I would suggest only using this if an empty cell is used for something not existing. For example, I have 75 total pieces. If I only have a code that is STRING_LENGTH(CODEBASESTRING) = 7, then the pieces past 7 are not included in the assembly. You should do the same for the code as well.
Cheers,
Jonathyn
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
@BenLoosli wrote:
Can you use a string parameter to drive your interchange instead of a floating number? That would make it cleaner and there would be no rounding. Possibly an integer may be an alterntive.
Geez I completely missed this reply...
You must have been typing faster than me.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
@pmajor wrote:
When I input a number that is greater than 16 digits, it begins to round. For example, I insert the following:
2222222222222222222222222.00000
and it automatically corrects it to:
2222222222222222184939520.000000
I don't understand why it is doing this. I need these numbers to be exact becuase it is a code that drives parameters based on the numbers 3 and 2.
For instance, if I put 232323223, an interchange will grab the part associated with each individual number. It is not being using for a dimensional value.
Thanks,
John
I would suggest using a string parameter for this since it is actually a part number and not a numeric value.