Precision when rounding BigDecimal is not working JBEDS
Issue
- When rounding a BigDecimal, instead of rounding and truncating the result it is just setting the decimal places after the rounding to zero so we end up with the same number of decimal places. For example, take the BigDecimal 1.123456789. If you round this to 2 decimal places I would expect the answer to be 1.12 with a length of 4, but JBEDS is returning 1.120000000 and length 11.
select convert('1.123456789', bigdecimal) a
, round(convert('1.123456789', bigdecimal),2) b
, length(round(convert('1.123456789', bigdecimal),2)) c
returns:
1.123456789 1.120000000 11
but should be:
1.123456789 1.12 4
Environment
- JBoss Enterprise Data Services (JBEDS) 5.2.0, 5.3.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
