Hello,
On 12/01/2008, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Sat, Jan 12, 2008 at 03:12:28PM +0000, Robert Swain wrote:
> > I also tested using float versus double for local variables using the
> > same technique (stopping when the current coefficients are the same as
> > the last iteration) and found the difference in the values reached
> > between float and double occurred after ~5 decimal places. I didn't
> > know if this was significant or not. So I've left this for the moment.
>> 5 correct decimal places:
> 65536 * 0.000009 = 0.589824
> so id say using floats is not exactly something id consider a good idea
> the doubles dont cost anything. While floats would limit the output to
> about 16bit precission.
I didn't think 5 d.p. would be acceptable. The actual differences were
small (the next decimal place was out by 1) but still, I've left it
using doubles.
> and patch ok
Applied with reference to this thread in the commit log message in
case anyone wants to see the discussion of the number of iterations.
Thanks for the review and discussion,
Rob