Wednesday, April 05, 2006

Ruby: Real number precision

If you do some things that involve float/real/double numbers in Ruby, you must lookout for one thing. For example if one write code like this:

#!/opt/local/bin/ruby

(0..5).each {|x|
puts x/2
}

the result will be:

0
0
1
1
2
2

As everyone can see those results are quite incorrect, cause Ruby assumes that x/2 is division of two integer numbers; hence, the result also must be integer. Such small thing, can be very unseen and hard bug to detect, especially if long, mathematical function is written.
Correct version of the above code is:

#!/opt/local/bin/ruby

(0..5).each {|x|
puts x.to_f/2
}

The result finally is ok:

0.0
0.5
1.0
1.5
2.0
2.5


I had this bug lately and it took me quite a time to find out why my program gives incorrect results.

No comments:

Post a Comment