Table of Contents
Advertisement
Precautions
Precautions when an input value of a single/double-precision real number is set using a programming tool are shown below.
■Single-precision real number
Because single-precision real number data are processed as the 32-bit single-precision in a programming tool, the number of
significant digits becomes approximately 7. An input value of the single-precision real number data exceeds 7 digits, 8th digit
is rounded. If the value after rounding exceeds the range of -2147483648 to 2147483647, an operation error occurs.
Example 1: When '2147483647' is set for the input value
8th digit '6' is rounded.
The value is handled as '2147484000'.
Example 2: When 'E1.1754943562' is set for the input value
8th digit '3' is rounded.
The value is handled as 'E1.175494'.
■Double-precision real number
Because double-precision real number data are processed as the 64-bit double-precision in a programming tool, the number
of significant digits becomes approximately 15. An input value of the double-precision real number data exceeds 15 digits,
16th digit is rounded. If the value after rounding exceeds the range of -2147483648 to 2147483647, an operation error occurs.
Example 1: When '2147483646.12345678' is set for the input value
16th digit '6' is rounded.
The value is handled as '2147483646.12346'.
Example 2: When 'E1.7976931348623157+307' is set for the input value
16th digit '5' is rounded.
The value is handled as 'E1.79769313486232+307'.
3 CONFIGURATION OF INSTRUCTIONS
3.2 Designating Data
3
89
Advertisement
Table of Contents
