Table of Contents
Advertisement
Appendix A: Accuracy calculations
Calculating amps accuracy
From the specifications, amps is calculated as follows:
Accuracy
= ± (% rdg + counts)
The following example shows how to compute accuracy for the 20 mA range:
Assume that the current you are measuring is reading exactly 10.0000 mA on the 20 mA range.
From the specifications:
Accuracy
= ± (0.1 % of 10 mA + 5 counts)
= ± (0.01 mA + 5 counts)
= ± (0.01 mA + 0.0005 A)
= ± 0.0105 mA
Five counts on the 20 mA range equals 00.0005 mA.
Therefore, the accuracy range for the 10.0000 mA reading is 9.9895 mA to 10.0105 mA.
Calculating ohms accuracy
The following information shows how to calculate auto V-source ohms accuracy.
From the specifications, accuracy is calculated as follows:
Accuracy
= ± (% rdg + counts)
The following example shows how to compute auto V-source ohms accuracy for the 2 MΩ range:
Assume that the resistor you are measuring is reading exactly 1.00000 MΩ on the 2 MΩ range.
From the specs:
Accuracy
= ± (0.125 % of 1 MΩ + 1 count)
= ± (1250 Ω + 1 count)
= ± (1250 Ω + 10 Ω)
= ±1260 Ω
One count on the 2 MΩ range (0.00001 MΩ) equals 10 Ω
Therefore, the accuracy range for the 1.00000 M auto V-source reading is 0.99874 MΩ to
1.00126 MΩ.
A-2
Model 6517B Electrometer Reference Manual
6517B-901-01 Rev. C / August 2015
Advertisement
Table of Contents
