Renesas H8S/2319 series Hardware Manual page 566

• Quantization error
The deviation inherent in the A/D converter, given by 1/2 LSB (see figure 14-8).
• Nonlinearity error
The error with respect to the ideal A/D conversion characteristic between the zero voltage and
the full-scale voltage. Does not include the offset error, full-scale error, or quantization error.
• Absolute precision
The deviation between the digital value and the analog input value. Includes the offset error,
full-scale error, quantization error, and nonlinearity error.
Figure 14-8 A/D Conversion Precision Definitions (1)
Rev. 5.00, 12/03, page 536 of 1088
Digital output
Ideal A/D conversion
characteristic
111
110
101
100
011
010
001
000
1 2
1024 1024
Quantization error
1022 1023
FS
1024 1024
Analog
input voltage