Dc Specifications; Table 6-1 Dc Resolution, Full Scale Reading, And Accuracy - Agilent Technologies U3402A User's Manual And Service Manual

6 Specifications and Characteristics

DC Specifications

Table 6-1 DC resolution, full scale reading, and accuracy [±(% of reading + count)]
Function Rate
Slow
Voltage Med
Fast
90
Range Resolution
120.000 mV 0.001 mV
1.20000 V 0.00001 V
12.0000 V 0.0001 V
120.000 V 0.001 V
1000.00 V 0.01 V
400.00 mV 0.01 mV
4.0000 V 0.0001 V
40.000 V 0.001 V
400.00 V 0.01 V
1000.0 V 0.1 V
400.0 mV 0.1 mV
4.000 V 0.001 V
40.00 V 0.01 V
400.0 V 0.1 V
1000 V 1 V
Accuracy
Maximum
(One year; 23ºC
reading
± 5ºC)
119.999 ±0.012% + 8
1.19999 ±0.012% + 5
11.9999 ±0.012% + 5
119.999 ±0.012% + 5
[3]
1000.00 ±0.012% + 5
399.99 ±0.012% + 5
3.9999 ±0.012% + 5
39.999 ±0.012% + 5
399.99 ±0.012% + 5
[3]
1000.0 ±0.012% + 5
399.9 ±0.012% + 2
3.999 ±0.012% + 2
39.99 ±0.012% + 2
399.9 ±0.012% + 2
[3]
1000 ±0.012% + 2
Typical input Burden voltage
[1]
impedance
[2]
10.0 MW
10.0 MW
11.1 MW
10.1 MW
10.0 MW
10.0 MW
11.1 MW
10.1 MW
10.0 MW
10.0 MW
10.0 MW
11.1 MW
10.1 MW
10.0 MW
10.0 MW
U3402A User's and Service Guide
[4]
—
—
—