Fluke 2680A Service Manual page 192

268XA
Service Manual
CAL
Constant
Number
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
42
44
46
48
50
52
54
56
58
60
*CFC = crystal frequency correction
6-46
Table 6-13. A/D Module Calibration Constants
Function Range
Function
V dc 90 mV
V dc 90 mV
V dc 300 mV
V dc 300 mV
V dc 3 V
V dc 3 V
V dc 30 V
V dc 30 V
V dc 50 V (FAI)
150/300 V (PAI)
V dc 50 V (FAI)
150/300 V (PAI)
V dc 750 mV
V dc 750 mV
V ac 300 mV
V ac 300 mV
V ac 3 V
V ac 3 V
V ac 30 V
V ac 30 V
V ac N/A (FAI)
150/300 V (PAI)
V ac N/A (FAI)
150/300 V (PAI)
Resistance 300
Resistance 300
Resistance 3 k
Resistance 3 k
Resistance 30 k
Resistance 30 k
Resistance 300 k
Resistance 300 k
Resistance 3 M
Resistance 3 M
Frequency All
Minimum
Comment Allowable Value
Gain +0.95000E+0
Offset -0.00090E+0
Gain +0.95000E+0
Offset -0.00300E+0
Gain +0.95000E+0
Offset -0.03000E+0
Gain +0.95000E+0
Offset -0.30000E+0
Gain +0.95000E+0
Offset -0.50000E+0
-3.00000E+0
Gain +0.95000E+0
Offset -0.00750E+0
Gain +0.90000E+0
Offset -0.00300E+0
Gain +0.90000E+0
Offset -0.03000E+0
Gain +0.90000E+0
Offset -0.30000E+0
Gain +0.90000E+0
Offset -3.00000E+0
Gain +0.95000E+0
Offset -3.00000E+0
Gain +0.95000E+0
Offset -3.00000E+1
Gain +0.95000E+0
Offset -3.00000E+2
Gain +0.95000E+0
Offset -3.00000E+3
Gain +0.95000E+0
Offset -3.00000E+4
CFC* +0.95000E+0
Maximum
Allowable
Value
+1.05000E+0
+0.00090E+0
+1.05000E+0
+0.00300E+0
+1.05000E+0
+0.03000E+0
+1.05000E+0
+0.30000E+0
+1.05000E+0
+0.50000E+0
+3.00000E+0
+1.05000E+0
+0.00750E+0
+1.10000E+0
+0.00300E+0
+1.10000E+0
+0.03000E+0
+1.10000E+0
+0.30000E+0
+1.10000E+0
+3.00000E+0
+1.05000E+0
+3.00000E+0
+1.05000E+0
+3.00000E+1
+1.05000E+0
+3.00000E+2
+1.05000E+0
+3.00000E+3
+1.05000E+0
+3.00000E+4
+1.05000E+0