Why is BigDecimal returning a weird value?

Question

I am writing code that will deal with currencies, charges, etc. I am going to use the BigDecimal class for math and storage, but we ran into something weird with it.

This statement:

1876.8 == BigDecimal('1876.8')

returns false.

If I run those values through a formatting string "%.13f" I get:

"%.20f" % 1876.8 => 1876.8000000000000
"%.20f" % BigDecimal('1876.8') => 1876.8000000000002

Note the extra 2 from the BigDecimal at the last decimal place.

I thought BigDecimal was supposed to counter the inaccuracies of storing real numbers directly in the native floating point of the computer. Where is this 2 coming from?

Answer 1

You are right, BigDecimal should be storing it correctly, my best guess is:

• BigDecimal is storing the value correctly
• When passed to a string formatting function, BigDecimal is being cast as a lower precision floating point value, creating the ...02.
• When compared directly with a float, the float has an extra decimal place far beyond the 20 you see (classic floats can't be compared behavoir).

Either way, you are unlikely to get accurate results comparing a float to a BigDecimal.

Answer 2

It won't give you as much control over the number of decimal places, but the conventional format mechanism for BigDecimal appears to be:

a.to_s('F')

If you need more control, consider using the Money gem, assuming your domain problem is mostly about currency.

gem install money

Answer 3

Don't compare FPU decimal string fractions for equality

The problem is that the equality comparison of a floating or double value with a decimal constant that contains a fraction is rarely successful.

Very few decimal string fractions have exact values in the binary FP representation, so equality comparisons are usually doomed.^*

To answer your exact question, the 2 is coming from a slightly different conversion of the decimal string fraction into the Float format. Because the fraction cannot be represented exactly, it's possible that two computations will consider different amounts of precision in intermediate calculations and ultimately end up rounding the result to a 52-bit IEEE 754 double precision mantissa differently. It hardly matters because there is no exact representation anyway, but one is probably more wrong than the other.

In particular, your 1876.8 cannot be represented exactly by an FP object, in fact, between 0.01 and 0.99, only 0.25, 0.50, and 0.75 have exact binary representations. All the others, include 1876.8, repeat forever and are rounded to 52 bits. This is about half of the reason that BigDecimal even exists. (The other half of the reason is the fixed precision of FP data: sometimes you need more.)

So, the result that you get when comparing an actual machine value with a decimal string constant depends on every single bit in the binary fraction ... down to 1/2⁵² ... and even then requires rounding.

If there is anything even the slightest bit (hehe, bit, sorry) imperfect about the process that produced the number, or the input conversion code, or anything else involved, they won't look exactly equal.

An argument could even be made that the comparison should always fail because no IEEE-format FPU can even represent that number exactly. They really are not equal, even though they look like it. On the left, your decimal string has been converted to a binary string, and most of the numbers just don't convert exactly. On the right, it's still a decimal string.

So don't mix floats with BigDecimal, just compare one BigDecimal with another BigDecimal. (Even when both operands are floats, testing for equality requires great care or a fuzzy test. Also, don't trust every formatted digit: output formatting will carry remainders way off the right side of the fraction, so you don't generally start seeing zeroes, you will just see garbage values.)

---

^{*The problem: machine numbers are x/2n, but decimal constants are x/(2n * 5m). Your value as sign, exponent, and mantissa is the infinitely repeating 0 10000001001 1101010100110011001100110011001100110011001100110011... Ironically, FP arithmetic is perfectly precise and equality comparisons work perfectly well when the value has no fraction.}

Answer 4

as David said, BigDecimal is storing it right

 p (BigDecimal('1876.8') * 100000000000000).to_i

returns 187680000000000000

so, yes, the string formatting is ruining it

Answer 5

If you don't need fractional cents, consider storing and manipulating the currency as an integer, then dividing by 100 when it's time to display. I find that easier than dealing with the inevitable precision issues of storing and manipulating in floating point.

Answer 6

On Mac OS X, I'm running ruby 1.8.7 (2008-08-11 patchlevel 72) [i686-darwin9]

irb(main):004:0> 1876.8 == BigDecimal('1876.8') => true

However, being Ruby, I think you should think in terms of messages sent to objects. What does this return to you:

BigDecimal('1876.8') == 1876.8

The two aren't equivalent, and if you're trying to use BigDecimal's ability to determine precise decimal equality, it should be the receiver of the message asking about the equality.

For the same reason I don't think formatting the BigDecimal by sending a format message to the format string is the right approach either.