Calculation Ranges, Number Of Digits, And Precision - Casio fx-100AU PLUS User Manual

29: hp ' kW
m ' J
33: kgf
•
37: °F ' °C
Conversion formula data is based on the "NIST Special Publication 811
(2008)".
Note: The J'cal command performs conversion for values at a temperature
of 15°C.
Calculation Ranges, Number of Digits,
and Precision
The calculation range, number of digits used for internal calculation, and
calculation precision depend on the type of calculation you are performing.
Calculation Range and Precision
Calculation Range
Number of Digits for Internal
Calculation
Precision
Function Calculation Input Ranges and Precision
Functions
x
sin
x
cos
x
tan
x
–1
sin
x
–1
cos
x
tan –1
x
sinh
x
cosh
x
–1
sinh
x
–1
cosh
x
tanh
x
–1
tanh
x x
log /ln
30: kW ' hp
34: J ' kgf
38: °C ' °F
| |
x
DEG
0
| |
x
RAD
0
| |
x
GRA
0
| |
x
DEG
0
| |
x
RAD
0
| |
x
GRA
0
DEG
Same as sin
RAD
Same as sin
GRA
Same as sin
| |
x
0 1
| | 9.999999999 × 10
x
0
| |
x
0 230.2585092
| | 4.999999999 × 10
x
0
4.999999999 × 10
x
1
| | 9.999999999 × 10
x
0
| | 9.999999999 × 10
x
0
9.999999999 × 10
x
0
31: kgf/cm
m 35: lbf/in
•
39: J ' cal
± 1 × 10 to ± 9.999999999 × 10
–99
15 digits
In general, ± 1 at the 10th digit for a single
calculation. Precision for exponential
display is ± 1 at the least significant digit.
Errors are cumulative in the case of
consecutive calculations.
Input Range
9 × 10
9
157079632.7
1 × 10
10
9 × 10
9
157079632.7
1 × 10
10
, except when |
x
, except when |
x
, except when |
x
99
99
99
99
–1
99
E-30
' Pa 32: Pa ' kgf/cm
2
' kPa 36: kPa ' lbf/in
2
40: cal ' J
| = (2 –1) × 90.
x n
| = (2 –1) × π /2.
x n
| = (2 –1) × 100.
x n
2
2
99
or 0