(this is just part of the comments message but a part I wanted to check
up on)
> -------- Original Message --------
> Subject: avg testcase issues (was: Re: syntax error in testcase manifest)
...
> From: Jeen Broekstra <jeen.broekstra@gmail.com>
...
> 1. testcase aggregates/agg-avg-01 ("AVG")
>
> This testcase currently assumes a particular precision in xsd:decimal
> division, which is not mandated by the XPath definition. XPath defines
> that in xsd:decimal operations, rounding is implementation dependent
> (see http://www.w3.org/TR/xpath-functions/#op.numeric, near the end of
> the section).
>
> In the test, five numbers of type xsd:decimal are averaged (1.0, 2.0,
> 3.5, 2.2, 2.2). The average is computed by executing a op:numeric-divide
> on the sum of the five decimals: 11.1/5. The result of this is, as the
> testcase defines, 2.22. However, this result increases precision to 2
> decimals (note that the precision of the input is 1 decimal).
>
> While this is not wrong, it is not the *only* correct answer: an
> implementation that would round or truncate to 1 decimal and return 2.2
> as the answer would IMHO also perform the operation correctly.
>
> To fix this, I would suggest that the testcase numbers are adapted so
> that the result of the division when not rounded is of the same
> precision as the input. Alternatively, change the test case to work on
> floats or doubles, as division behavior is more precisely defined for
> those datatypes.
The issue seems to be that reading "2.0" is implying a precision of one
decimal place in the decimal. This isn't true - XSD decimal must
support at least 18 digits of precision and precision is not a facet of
the lexical space.
http://www.w3.org/TR/xmlschema-2/#decimal
"""
Note: All ·minimally conforming· processors ·must· support decimal
numbers with a minimum of 18 decimal digits (i.e., with a ·totalDigits·
of 18)
"""
1.0, 2.0 etc are at least 18 digits of precision and the way the decimal
is written is not relevant. i.e. In XSD, 2.00 is not promising more
precision than 2.0 - they are different ways to write the same value down.
"""
Precision is not reflected in this value space; the number 2.0 is not
distinct from the number 2.00.
"""
which is a little different from Java BigDecimal where scale is a factor
in BigDecimal.equals or some scientific styles where the number of
places does imply accuracy.
Andy