| Georgi Guninski on Fri, 16 Feb 2024 18:01:14 +0100 |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
| Re: Numerical instability in extracting base B digits of the fractional part of real |
Thanks. I think you are not computing the same algorithm, since you hardcode $d=3$: ? A=0.33;B=10; ? C=A*B;d=floor(C);A=C-d %98 = 0.30000000000000000000000000000000000000 ? C=A*B;d=floor(C);A=C-d %99 = 0.99999999999999999999999999999999999998 ? d %100 = 2