APPENDIX
Function
n > 0
|Σx| < 1 × 10
|(Σx) (Σy)| < 1 × 10
0 < |Σx
b
|Σxy –
|bx| < 1 × 10
a
|y – bx| < 1 × 10
|bx| < 1 × 10
y'
|a + bx| < 1 × 10
|y – a| < 1 × 10
x'
|
In principle, the residual for calculations is ±1 of the last digit (in case of exponential
display, the residual is ±1 of the last digit of the mantissa display).
However, the residual increases in continuous calculations due to accumulation of each
residual (this is the same for squares (a
calculations are performed internally).
Moreover, errors will accumulate and become larger in the vicinity of inflection points
and singular points of functions.
Calculation range
50
100
| < 1 × 10
2
–
| < 1 × 10
100
< 1 × 10
100
100
100
100
100
100
| < 1 × 10
100
b
100
) and square roots (
280
Notes
Regression calcula-
tions excluding 2nd,
3rd and 4th degree
polynomials.
Same as above.
Same as b for other.
) where continuous