Measuring Range And Measuring Accuracy - Omron K7GE-MG User Manual

4-3 Measuring Range and Measuring
Accuracy
The measuring range is between 0.1 and 99.9
MΩ.
The alarm judgment is treated as 0 MΩ for inputs
less than 0.1 MΩ and as 99.9 MΩ for inputs
greater than 99.9 MΩ.
The display shows 0.0 for inputs less than 0.1 MΩ
and flashes 99.9 for inputs greater than 99.9 MΩ.
The measuring accuracy is as follows:
±5% rdg. ±1 digit (at ambient temperature -10 to 55°C and ambient humidity 25% to 65%)
"rdg." is the abbreviation for "reading" and refers to the read value (displayed numeric value). ±5% rdg.
indicates the following range:
(Read value) × (1 - 0.05) to (Read value) × (1 + 0.05)
"digit" is the minimum resolution of the read value. In the K7GE-MG, the minimum resolution of the read
value is 0.1 MΩ. Therefore, ±1 digit becomes ±0.1 MΩ.
Thus, for example, if the read value is assumed to
be 10.0, the upper and lower limit values will
become as follows.
Lower limit value: 10.0 × (1 - 0.05) - 0.1 = 9.4 MΩ
Upper limit value: 10.0 × (1 + 0.05) + 0.1 = 10.6
MΩ.
That is, the true value will be in the range of 9.4 to
10.6 MΩ.
Insulation Resistance Monitoring Device User's Manual (N224)
4 Functions Related to Measurement
Display value
Flashes as
99.9
99.9
10.0
1.0
0.1
0.0
0.1 M 1 M
True
value
10.6 M
10.5 M
10.0 M
9.5 M
9.4 M
5.0 M 10.0 M
4
Measurement
10 M 99.9 M
value
+1 digit
±5% rdg.
-1 digit
15.0 M
Read value
(display value)
4 - 21