Siemens SINUMERIK 840D sl Function Manual page 321

The input and display resolution is determined by the specified operator panel front used,
whereby the display resolution for position values with the machine data:
MD9004 $MM_DISPLAY_RESOLUTION (display resolution)
can be changed.
The machine data:
MD9011 $MM_DISPLAY_RESOLUTION_INCH (display resolution for INCH measuring
system)
can be used to configure the display resolution for position values with inch setting.
This allows you to display up to six decimal places with the inch setting.
For the programming of part programs, the input resolutions listed in the Programming Guide
apply.
The desired computational resolution is defined using the machine data:
MD10200 $MN_INT_INCR_PER_MM (computational resolution for linear positions)
and
MD10210 $MN_INT_INCR_PER_ DEG (computational resolution for angular positions).
It is independent of the input/display resolution but should have at least the same
resolution.
The maximum number of places after the decimal point for position values, velocities, etc., in
the part program and the number of places after the decimal point for tool offsets, zero offsets,
etc. (and therefore also for the maximum possible accuracy) is defined by the computational
resolution.
The accuracy of angle and linear positions is limited to the computational resolution by rounding
the product of the programmed value with the computational resolution to an integer number.
To make the rounding clear, powers of 10 should be used for the calculation resolution.
Example of rounding:
Computational resolution: 1000 incr./mm
Programmed path: 97.3786 mm
Effective value: 97.379 mm
Example of programming in the
All the linear axes of a machine are to be programmed and traversed within the range of values
0.1 to 1000 μm.
⇒ In order to position accurately to 0.1 μm, the computational resolution must be set to ≥ 10
incr./mm.
⇒ MD10200 $MN_INT_INCR_PER_MM = 10000 [incr./mm]:
Basic Functions
Function Manual, 01/2015, 6FC5397-0BP40-5BA2
G2: Velocities, setpoint / actual value systems, closed-loop control
/ μm range:
1
10
7.2 Velocities, traversing ranges, accuracies
4
321