Table of Contents
Advertisement
Keys:
Å
70
ˆ
º %
{
}
Limitations on Precision of Data
Since the calculator uses finite precision (12 to 15 digits), it follows that there are
limitations to calculations due to rounding. Here are two examples:
Normalizing Close, Large Numbers
The calculator might be unable to correctly calculate the standard deviation and
linear regression for a variable whose data values differ by a relatively small
amount. To avoid this, normalize the data by entering each value as the difference
from one central value (such as the mean). For normalized x–values, this difference
must then be added back to the calculation of
also be adjusted. For example, if your x–values were 7776999, 7777000, and
7777001, you should enter the data as –1, 0, and 1; then add 7777000 back to
x
x ˆ
. For b, add back 7777000 × m. To calculate
and
an x–value that is less 7777000.
Similar inaccuracies can result if your x and y values have greatly different
magnitudes. Again, scaling the data can avoid this problem.
Effect of Deleted Data
¹ -
Executing
generated in the statistics registers by the original data values. This difference is
not serious unless the incorrect data have a magnitude that is enormous compared
with the correct data; in such a case, it would be wise to clear and reenter all the
data.
11–10
Statistical Operations
File name 33s-E-Manual-1008-Publication(1st).doc
Printed Date : 2003/10/8
Display:
_
ˆ
ˆ
does not delete any rounding errors that might have been
Size : 13.7 x 21.2 cm
Description:
Enters hypothetical x–value.
The predicted yield in tons per
hectare.
y ˆ
x
x ˆ
and
, and
y ˆ
, be sure to supply
Page : 386
and b must
Advertisement
Table of Contents
