Level Oo : Overflow/Underflow; Level 1: Correctly Rounded, Or Nearly So - HP -15C Advanced Functions Handbook

Appendix: Accuracy of Numerical Calculations
179
Level °°: Overflow/Underflow
Results which would lie closer to zero than 10"" underflow quietly
to zero. Any result that would lie beyond the overflow thresholds
±9.999999999 X 10" is replaced by the nearest threshold, and then
flag 9 is set and the display blinks. (Pressing [ O N | [ O N | or [CF] 9 or H
will clear flag 9 and stop the blinking.) Most functions that result
in more than one component can tolerate overflow/ underflow in
one component without contaminating the other; examples are
r+R } , r+P"|, complex arithmetic, and most matrix operations. The
exceptions are matrix inversion (1 1/x| and |T)), I MATRIX] 9
(determinant), and| LR."|.
Level 1 : Correctly Rounded, or Nearly So
Operations that deliver "correctly rounded" results whose error
cannot exceed Vz unit in their last (10th) significant digit include
the real algebraic operations H , H > B > B > H > QEJ . 1
1/
*l >
an
d f%1 ,
the complex and matrix operations [+] and [-1, matrix by scalar
operations [x] and {±\g division by a matrix), and | -M-I.MSl .
These results are the best that 10 significant digits can represent,
as are familiar constants QT], 1 [e*1, 2 [TNl,
1P
| L N | , 1 |-»RAD|, and
many more. Operations that can suffer a slightly larger error, but
still significantly smaller than one unit in the 10th significant digit
of the result, include | A% | , |-»H| , |-»RAD| , |-»DEG| , | Py.x \ and |Cy,x| ;
[LNl , J L O G | , 1 10* | , and | T A N H | for real arguments; |-»P| , | SIN'
1
1 , | COS"
1
1 ,
| TAN
1
1 , | SIMM'
1
1, | COSH"
1
1, and JTANH"
1
! for real and complex
arguments; I ABS|, \lx~\, and 1 1/x| for complex arguments; matrix
norms | MATRIX | 7 and [ M A T R I X ] 8; and finally | SIN |, |COS|, and |TAN|
for real arguments in Degrees and Grads modes (but not in
Radians mode — refer to Level 2, page 184).
A function that grows to °° or decays to 0 exponentially fast as its
argument approaches ±°° may suffer an error larger than one unit
in its 10th significant digit, but only if its magnitude is smaller
than 10~
20
or larger than 10
20
; and though the relative error gets
worse as the result gets more extreme (small or large), the error
stays below three units in the last (10th) significant digit. The
reason for this error is explained later. Functions so affected are
[e*~\, [y*1. fxTI (for noninteger x), | S I N H | , and |COSH| for real
arguments. The worst case known is 3
201
, which is calculated as
7.968419664 X 10
95
. The last digit 4 should be 6 instead, as is the
case for 7.29
33
'
5
, calculated as 7.968419666 X 10
28
.