Summary of Contents for Intel RealSense D400 Series
- Page 1 ® Intel RealSense™ Depth Module D400 Series Custom Calibration Revision 1.1.0 January 2018...
- Page 2 PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD...
-
Page 3: Table Of Contents
USB ..................17 3.1.5 PC ..................17 Software ....................17 3.2.1 Custom Calibration Sample Application ........17 3.2.2 Intel® RealSense™ Calibration Tool and API ....... 18 3.2.3 Intel® RealSense™ SDK ............19 3.2.4 OpenCV 3.3.0 ................. 19 3.2.5 Glut Library ................20 Calibrating Device with Custom Calibration Sample Application...... - Page 4 Calculating Depth Camera Calibration with OpenCV ........46 Calculating RGB Camera Calibration with OpenCV ........47 Calculating RGB Camera Calibration Extrinsics with OpenCV ....... 47 Writing Calibration Parameters ............... 48...
- Page 5 Tables Table 2-1. Frame Formats Used in Custom Calibration .......... 11 Table 3-1. Intel® RealSense™ Calibration API Resources ........18 Table 3-2. Intel® RealSense™ SDK Resources ............ 19 Table 3-3. OpenCV 3.3.0 Resources ..............20 Table 3-4. OpenCV 3.3.0 Libraries Required for the Example ......... 20...
- Page 6 List of Figures Figure 2-1 Software Stack with Dynamic Calibration API and Calibration Apps ... 9 Figure 3-1 Hardware Setup ................13 Figure 3-2 D415 Device ..................14 Figure 3-3 8x7 60x60 mm Checker Calibration Target .......... 15 Figure 3-4 Tripod ..................... 16 Figure 4-1 Center Right Position ................
- Page 7 Revision History Document Revision Description Revision Date Number Number XXXXXX Initial Release 01/2018 Update with Calibration API 2.5.2.0 release 01/2018...
-
Page 8: Introduction
Purpose and Scope of This Document In order to operate Intel® RealSense™ D400 device efficiently and accurately, users need to make sure the device is well calibrated. The Intel supplied calibration tools including Intel® RealSense™ Dynamic Calibrator and OEM Calibration Tool for Intel® RealSense™... -
Page 9: Overview
Overview Overview Calibration API and Calibration Data Read/Write/Restore Intel provides a software interface in Calibration API to enable user uploading those calibration parameters to Intel® RealSense™ D400 devices and read the parameter back from device: WriteCustomCalibrationParameters – write parameters to device ... -
Page 10: Calibration Parameters
EXTRINSICS. The user custom algorithm is free to optimize all these parameters. The sample in this document optimizes both intrinsic and extrinsic parameters. The following table shows how the various calibration tools would impact calibration parameters for Intel® RealSense™ D400 series depth cameras. Factory Technician... -
Page 11: Frame Formats Used In Custom Calibration
Calibration Sample App uses these formats. Table 2-1. Frame Formats Used in Custom Calibration Format Used Comment D400 Left and Right Sensors: Intel® RealSense™ Camera D400, D410, D415 1920x1080 @ 15 FPS D410 (16-bit) D420 Left and Right Sensors: Intel® RealSense™ Camera D430 D420, D435... - Page 12 Overview also not be accurate if the camera or board is handheld (or not completely still) as there is no framesync in calibration mode.
-
Page 13: Setup
Setup Setup This section describes the required hardware and software setup for running the Custom Calibration Sample Application to calibrate a device and developing a Custom Calibration Application. Hardware The hardware required including a calibration target, D400 series device to be calibrated, USB cable, and a Windows 10 computer. -
Page 14: Device
Setup 3.1.1 Device Intel® RealSense™ D415 device as shown below is used to show case the custom calibration process. All D400 series devices can use the same process. Figure 3-2 D415 Device 3.1.2 Target The target used in this custom calibration example is an 8x7 checkerboard with a 60x60mm checker size. -
Page 15: Tripod
Setup Figure 3-3 8x7 60x60 mm Checker Calibration Target 3.1.3 Tripod Any medium sized tripod should be sufficient. In this example setup, we used a Manfrotto Compact Light Aluminum Tripod with ball head. The ball head makes adjustment to device orientation easier. -
Page 16: Figure 3-4 Tripod
Setup Figure 3-4 Tripod... -
Page 17: Usb
Custom Calibration Sample Application A CustomCalibrationSample.zip package is provided for the Custom Calibration Sample Application. It includes the binary executable and a calibration target file. The source code is included in Intel® RealSense™ Calibration API package. Unzip the package: CustomCalibrationSample |---------Bin |-------- CustomCalibrationSampleApp.exe... -
Page 18: Intel® Realsense™ Calibration Tool And Api
3.2.2 Intel® RealSense™ Calibration Tool and API The calibration example uses support functions provided by the Intel® RealSense™ Calibration API to store updated calibration parameters into the non-volatile storage on the Intel® RealSense™ D400 series modules. Required functions were first introduced in version 2.5.2.0 of the Dynamic Calibration API, so install the latest... -
Page 19: Intel® Realsense™ Sdk
3.2.3 Intel® RealSense™ SDK Install the latest release of the Intel® RealSense™ SDK so your custom calibration application can use LibRealSense to conveniently open and access Intel® RealSense™ D400 series modules. Table 3-2 contains pointers to the SDK homepage, GitHub* repository where you can download the latest release, and the SDK documentation. -
Page 20: Glut Library
Setup Table 3-3. OpenCV 3.3.0 Resources Resource OpenCV GitHub* Repository https://github.com/opencv/opencv OpenCV 3.3.0 Download https://github.com/opencv/opencv/releases/tag/3.3.0 Table 3-4. OpenCV 3.3.0 Libraries Required for the Example OpenCV 3.3.0 Library ittnotify.lib opencv_calib3d330.lib opencv_core330.lib opencv_features2d330.lib opencv_flann330.lib opencv_imgproc330.lib zlib.lib 3.2.5 Glut Library The FreeGLUT package is a convenient open source alternative to the OpenGL Utility Toolkit for displaying graphical output to the operator of the custom calibration application. -
Page 21: Calibrating Device With Custom Calibration Sample Application
Calibrating Device with Custom Calibration Sample Application Calibrating Device with Custom Calibration Sample Application Process Overview The general process to calibrate a device with the Custom Calibration Sample App involves running the app capturing images of the target from various viewpoints, the app optimizes the calibration parameters based on the captured images and writes the results to a XML file. -
Page 22: Capturing Images From 6 Viewpoints
RGB, only the left and right images display on the lower left corner. A few text messages are overlay on top of the images: On the top left corner, it display the device model name, for example, Intel RealSense 415. ... - Page 23 Calibrating Device with Custom Calibration Sample Application Viewpoints - the choice of the viewpoint is critical to the accuracy of the calibration results. A general rule is that the target in these 6 images combined should cover as much as possible of the entire field of view and from different angles and distances from the target.
- Page 24 Calibrating Device with Custom Calibration Sample Application Bad example - cover only portion of the target in any of the three imagers: Now, let’s capture the images. 4.3.2.1 Viewpoint #1 – Center Right Adjust the tripod and position the device on the center right facing directly to the target.
-
Page 25: Figure 4-1 Center Right Position
Calibrating Device with Custom Calibration Sample Application Figure 4-1 Center Right Position The viewpoint should looks like below with the target cover much of the right side the FOV. -
Page 26: Figure 4-2 Viewpoint #1
Calibrating Device with Custom Calibration Sample Application Figure 4-2 Viewpoint #1 Press enter to capture the image. If successful, the frame counter on the lower right corner will increase to 1/6 meaning 1 out of 6 images are captured. If it failed, a red error message will appear and user is directed to retake a viewpoint. -
Page 27: Figure 4-3 Center Right
Calibrating Device with Custom Calibration Sample Application Figure 4-3 Center Right... -
Page 28: Figure 4-4 Viewpoint #2
Calibrating Device with Custom Calibration Sample Application Figure 4-4 Viewpoint #2 Press enter to capture the image. If successful, the frame counter on the lower right corner will increase to 2/6 meaning 2 out of 6 images are captured. If it failed, a red error message will appear and user is directed to retake a viewpoint. -
Page 29: Figure 4-5 Left
Calibrating Device with Custom Calibration Sample Application Figure 4-5 Left Figure 4-6 Viewpoint #3... -
Page 30: Figure 4-7 Right
Calibrating Device with Custom Calibration Sample Application Press enter to capture the image. If successful, the frame counter on the lower right corner will increase to 3/6 meaning 3 out of 6 images are captured. If it failed, a red error message will appear and user is directed to retake a viewpoint. -
Page 31: Figure 4-8 Viewpoint #4
Calibrating Device with Custom Calibration Sample Application Figure 4-8 Viewpoint #4 Press enter to capture the image. If successful, the frame counter on the lower right corner will increase to 4/6 meaning 4 out of 6 images are captured. If it failed, a red error message will appear and user is directed to retake a viewpoint. -
Page 32: Figure 4-9 Top Looking Down
Calibrating Device with Custom Calibration Sample Application Figure 4-9 Top Looking Down... -
Page 33: Figure 4-10 Viewpoint #5
Calibrating Device with Custom Calibration Sample Application Figure 4-10 Viewpoint #5 Press enter to capture the image. If successful, the frame counter on the lower right corner will increase to 5/6 meaning 5 out of 6 images are captured. If it failed, a red error message will appear and user is directed to retake a viewpoint. -
Page 34: Figure 4-11 Bottom Looking Up
Calibrating Device with Custom Calibration Sample Application Figure 4-11 Bottom Looking Up Figure 4-12 Viewpoint #6... -
Page 35: Calibration Result
Calibrating Device with Custom Calibration Sample Application Press enter to capture the image. If successful, the frame counter on the lower right corner will increase to 6/6 meaning 6 out of 6 images are captured. If it failed, a red error message will appear and user is directed to retake a viewpoint. - Page 36 Calibrating Device with Custom Calibration Sample Application <param name = "PrincipalPointLeft"> <value>965.562</value> <value>542.603</value> </param> <param name = "DistortionLeft"> <value>0.09689</value> <value>-0.0235634</value> <value>-5.05513e-05</value> <value>0.000105855</value> <value>-0.640377</value> </param> <param name = "FocalLengthRight"> <value>1389.55</value> <value>1390.1</value> </param> <param name = "PrincipalPointRight"> <value>957.402</value> <value>553.108</value> </param> <param name = "DistortionRight"> <value>0.123289</value>...
-
Page 37: Updating Results To Device
<param name = "TranslationLeftRGB"> <value>14.6987</value> <value>-0.0644999</value> <value>0.448085</value> </param> </Config> Updating Results to Device We will use Intel.Realsense.CustomRW to read the CalibrationParameters.xml and write the parameters to the device. Checking the depth quality before and after updating the calibration data on the device. -
Page 38: Depth Quality Check Before Updating Calibration
Writing Optimized Calibration to Device Change directory to Calibration Tool API bin: Cd C:\CalibrationToolAPI\2.5.2.0\bin Dump the current parameters on the device into before.txt: Intel.Realsense.CustomRW.exe –r > before.txt Writing the parameters in the XML result file into device: Intel.Realsense.CustomRW.exe –w CalibrationParameters.xml... -
Page 39: Figure 4-14 Updating Calibration Parameters To Device
Calibrating Device with Custom Calibration Sample Application Figure 4-14 Updating Calibration Parameters to Device Dump the parameters from device into after.txt: Intel.Realsense.CustomRW –r > after.txt Compare the parameters changed:... -
Page 40: Depth Quality Check After Updating Calibration
Calibrating Device with Custom Calibration Sample Application Figure 4-15 Calibration Parameter Change 4.5.3 Depth Quality Check after Updating Calibration... - Page 41 Calibrating Device with Custom Calibration Sample Application...
-
Page 42: Developing Custom Calibration Application
Calibrating Device with Custom Calibration Sample Application. The sample app uses OpenCV algo as example. The general process is to configure the Intel® RealSense™ device for calibration, capture images from the camera module, perform the desired calibration computations with OpenCV, and to write the new calibration parameters back to the cameras using routines from the Intel®... - Page 43 Developing Custom Calibration Application |------- DSDynamicCalibrationAPI.dll |------- freeglut.dll |------- realsense2.dll |-------- Include |------- DSDynamicCalibration.h |------- DSOSUtils.h |------- DSShared.h |------------- lib |------- DSDynamicCalibrationAPI.lib |-------- librealsense |------- include |------- lib |-------- examples |------- CustomCalibration |-------- CalibrationManager.cpp |-------- CalibrationManager.h |-------- CustomCalibration.cpp |-------- CustomCalibration.h |-------- CustomCalibrationWrapper.cpp |-------- CustomCalibrationWrapper.h |-------- Rs400Dev.cpp...
-
Page 44: Calibration Mode Camera Configuration
Developing Custom Calibration Application examples contains all Calibration API examples and related files. The CustomCalibration sample app source is under CustomCalibration folder. OpenCV 3.3 is not included. To compile the CustomCalibration sample app, user will need to set it up so that the OpenCV header files and libraries is under CalibrationToolAPI\2.5.2.0. OpenCV setup is discussed in section 3.2.4 “OpenCV 3.3.0”... -
Page 45: Streaming Resolution And Format
Developing Custom Calibration Application m_rsDevice->SetAeControl(800); else m_rsDevice->SetAeControl(500); 5.2.3 Streaming Resolution and Format Calibration frame resolutions and formats are described in Table 2-1 Frame Formats Used in Custom Calibration, for example, in this sample, or D415 device, the resolution is 1920x1080 with frame rate of 15 fps. The L/R format is Y16 and the RGB format is YUY2. -
Page 46: Calculating Depth Camera Calibration With Opencv
Developing Custom Calibration Application // Find chessboard corners if (!findChessboardCorners(image, chessboardSize, corners, CALIB_CB_ADAPTIVE_THRESH | CALIB_CB_NORMALIZE_IMAGE | CALIB_CB_FILTER_QUADS)) return false; // Refine them cornerSubPix(image, corners, Size(11, 11), Size(-1, -1), TermCriteria(CV_TERMCRIT_EPS | CV_TERMCRIT_ITER, 30, 0.1)); return true; Calculating Depth Camera Calibration with OpenCV #include <opencv2/opencv.hpp>... -
Page 47: Calculating Rgb Camera Calibration With Opencv
Developing Custom Calibration Application // Calibrate the extrinsics between them return stereoCalibrate(corners3D, cornersLeft, cornersRight, Kl, Dl, Kr, Dr, Size(-1, -1), Rlr, Tlr, noArray(), noArray(), CV_CALIB_FIX_INTRINSIC | CV_CALIB_USE_INTRINSIC_GUESS, TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 300, DBL_EPSILON)); Calculating RGB Camera Calibration with OpenCV Note: Assumes good depth camera calibration double CalibrateRGBCamera(const vector<vector<Point2f>... -
Page 48: Writing Calibration Parameters
Developing Custom Calibration Application Writing Calibration Parameters A user custom calibration app can choose one of the two approaches to update the results to the device: To link to the WriteCustomCalibrationParameters and ReadCalibrationParameters and write directly to the device through the APIs. ... - Page 49 Developing Custom Calibration Application...