GCD not working with vector
Jan 06, 2010
03:00 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Jan 06, 2010
03:00 AM
GCD not working with vector
See the very last lines in the attached file.
gcd (greatest common denominator) works when I specifically enter the individual array values, but fails when I specify only the array.
Your assistance would be greatly appreciated!
Roy Jensen
gcd (greatest common denominator) works when I specifically enter the individual array values, but fails when I specify only the array.
Your assistance would be greatly appreciated!
Roy Jensen
Labels:
- Labels:
-
Other
2 REPLIES 2
Jan 06, 2010
03:00 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Jan 06, 2010
03:00 AM
It's a round-off problem, coupled with a strange bug. The vector "norm" does not contain integers. That should not be a problem, as long as the gcd function either accepts them as integers to within some tolerance, and then rounds them internally, of flags an error. With a list of scalars it seems to do that, but with a vector there are certain cases where it just gives a bogus answer.
Richard
Richard
Jan 10, 2010
03:00 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Jan 10, 2010
03:00 AM
