Hesai Pandora User Manual
  1. Manuals
  2. Brands
  3. Hesai Manuals
  4. Accessories
  5. Pandora
  6. User manual

Hesai Pandora User Manual

All-in-one sensing solution for autonomous driving
Hide thumbs
Table of Contents

Advertisement

Quick Links

www.hesaitech.com
100-en-1801A2
Pandora
All-in-One Sensing Solution
for Autonomous Driving
User's Manual
HESAI Wechat

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the Pandora and is the answer not in the manual?

Questions and answers

Related Manuals for Hesai Pandora

Summary of Contents for Hesai Pandora

  • Page 1 100-en-1801A2 Pandora All-in-One Sensing Solution for Autonomous Driving User’s Manual HESAI Wechat...
  • Page 2 All reasonable efforts have been made to ensure the accuracy of the information. However, Hesai cannot be held responsible for any errors. Hesai does not warrant the accuracy and reserves the right to make changes to the catalog and its functions at any time without...

  • Page 3: Table Of Contents

    27-30 Point Cloud Data Packet Absolute Time and Laser Firing Time Calculations Pandora Data Structure Appendix III 11-18 LiDAR Data Structure 31-38 PandarView 19-20 Camera Data Structure Appendix IV Analysis of Data 39-40 Pandora Projection for ROS Support and Contact...

  • Page 4: Introduction

    Introduction Pandora is an all-in-one sensor kit for environment sensing for self-driving cars. It integrates cameras, LiDAR and data processing ability into the same module, with advanced synchronization and calibration solutions. In addition to the specifications of Pandora, this manual also describes the mechanical installation, data outputs format, and GPS timestamp synchronization.

  • Page 5: Pandora-Lidar

    Pandora — LiDAR The upside of Pandora is a 40-channel mechanical LiDAR. It creates 3D image by 360° rotating through 40 laser diodes inside the housing. LiDAR’s unique channel distribution makes it more suitable for autonomous driving applications. 1.1.1 Operational Principles Distance Measurement: Time of Flight (ToF) A laser diode emits a beam of ultrashort pulse laser on to the object.
  • Page 6 1.1.3 Channel Distribution The vertical angular resolution is 0.33° between Channel 6 and Channel 30; The vertical angular resolution is 1° between Channel 1 and Channel 6, Channel 30 and Channel 40. Please see Appendix I for detailed channel distribution. Channel 1 + 7°...

  • Page 7: Pandora-Camera

    Pandora—Camera Pandora has four wide-angle lens cameras and one forward-facing color camera capturing 360° image around the car. Camera Black and White：129° Color：52° LiDAR Figure 1.5 Coverage of Cameras 360° Type Resolution Color Camera 1280*720 52° (H), 28.6° (V), 61° (D）...

  • Page 8: Camera And Lidar Synchronization And Calibration

    Camera and LiDAR Synchronization and Calibration Synchronization Calibration Pandora controls the motor rotating and laser firing time of the Based on the calibration between point cloud and images, an LiDAR, and at the same time, LiDAR controls the exposure time accurate mapping between point cloud and camera pixel can be and frame rate of the cameras.

  • Page 9: Specifications

    -16°~7° Operating Temperature -10 ℃~60 ℃ Operating Voltage 9 V~32 V 0.33°（-6° to +2°）； Angular Resolution (Vertical) 1°（-16° to -6°，+2° to +7°） Weight 2.1 kg Return Mode Dual Return Enclosure Level IP66 Table 1.2 Pandora Specifications - 0 6 -...

  • Page 10: Installation

    Installation Pandora has already finished calibration before delivery. The unit is easy to install and plug-and-play. Please refer to chapter 2 for details on mechanical dimension, interfaces and connection methods. Mechanical Installation 270° 2-φ4mm 5.5mm FORφ4mm PINS φ118（0D） Reference Center φ88.9...

  • Page 11: Interfaces

    Interfaces Power Light Figure 2.3 Pandora Interfaces GPS port pin number from left to right is 1 to 6, and the specific GPS Port definition of each pin is shown as follows: Connector type: JST SM06B-SRSS-TB Pin No. Direction Pin Description...

  • Page 12: How To Connect

    Web control can be used to set parameters of Pandora before using. For more on web control, see Chapter 4. Users can use PandarView to quickly view or record point cloud data captured by Pandora. For more on PandarView installation and usage, see Appendix III PandarView.
  • Page 13 2.3.2 IP Configuration To receive data on your PC, please set the PC IP address to 192.168.20.100. Ubuntu-16.04: The IP address can be configured on the terminal by using the ifconfig command: ~$ sudo ifconfig enp0s20f0u2 192.168.20.100 Replace enp0s20f0u2 with the local network port name. Windows: 1) Open the Network Sharing Center, click on “Ethernet”.

  • Page 14: Pandora Data Structure

    Pandora Data Structure Users can receive both point cloud data from LiDAR and imaging data from cameras through Ethernet Cable. The data communication protocol for LiDAR is UDP protocol, while that for camera is TCP protocol. LiDAR Data Structure The communication protocol for data output of LiDAR is Gigabit Ethernet UDP/IP. The output data includes Point Cloud Data Packet and GPS Data Packet.
  • Page 15 Length 2 bytes (0x04F6, represents 1270 bytes), checksum 2 bytes Table 3.1 Definition of Point Cloud Data Packet Ethernet Header A Pandora has a unique MAC address. The destination IP address is 0xFF and in broadcast form. The default source IP address is 192.168.20.51.
  • Page 16 Point Cloud Data——UDP Data The UDP Data of Pandora has a 1262 bytes payload consisting of 1240 bytes ranging data and 22 bytes additional information. All the multi-byte values are the unsigned type and in Little Endian format. Ranging Data 1240 bytes (10 blocks) ······...
  • Page 17 Additional Information: 22 bytes Reserved 8 bytes reserved data, meaningless Motor Speed 2 bytes speed_2_bytes [15:0] = speed (RPM) GPS Timestamp 4 bytes the packing time of this data packet, the unit is 1 μs, value range 0 μs-1 s Return Mode Information 1 byte the strongest return (0x37), the last return (0x38), dual return (0x39)
  • Page 18 After the GPS module data has been sent to Pandora, the local 1 Hz signal of Pandora will be locked at PPS signal. The rising edge of the internal 1Hz signal will trigger a GPS Data Packet. Meantime, the GPS time data will be reset to actual GPS time and GRPMC information will be the original data sent by GPS module.
  • Page 19 The destination IP address is 0xFF and in broadcast form. The default source IP address is 192.168.20.51. Taking “Internet Protocol (20 bytes)” as an example, it is described as follows: Figure 3.3 GPS Data Packet Ethernet Header Internet Protocol Illustration GPS Data Packet - UDP Data UDP Data of GPS Data Packet consists of 18 bytes time data and 494 bytes additional information.
  • Page 20 Example of GPS Data Packet UDP Data Analysis Figure 3.4 GPS Data Packet UDP Data Illustration Date： Year: 0x37, 0x31, convert ASCII code to '7', '1'; means 17 Month: 0x32, 0x31, convert ASCII code to '2', '1'; means 12 Day: 0x30, 0x32, convert ASCII code to '0', '2'; means 20 Time：...
  • Page 21 <12> Mode (only on version NMEA0183 3.00, A=Automatic Positioning, D=Differential, E=Estimation, N=Invalid Data) The GPS interface of Pandora is compatible with a variety of data formats. The external GPS module GPRMC data format needs to meet the following conditions: the data in <01> is the hour, minute, and second information; the data in <09> is the date information.

  • Page 22: Camera Data Structure

    3.2 Camera Data Structure The communication protocol of cameras in Pandora is TCP protocol and the default port is 9870. The imaging data of each camera consists of two parts: Header and Payload. The Header includes SOP, Pic id, Type, Width, Height, Timestamp, Length, Total Length, Position, and UTC Time.

  • Page 23: Analysis Of Data

    The year of the real time can be calculated by 2000+(year in UTC Time). For example, year 18 (UTC time) means year 2018 in real time. After TCP protocol has established to the port for camera in Pandora, the “HEARTBEAT” packet at regular intervals are required to keep the connection, otherwise Pandora will drop the connection on its own.

  • Page 24: Web Control

    Web Control Web Control can be used to set parameters of Pandora, check device info, and upgrade. Please connect Pandora to the computer using ethernet cable and set IP address to 192.168.20.100. 4.1 Open Web Control After setting, open browser and type URL: 192.168.20.51/index.html to enter the web control homepage.

  • Page 25: Setting

    4.2 Setting 1) Pandora supports both broadcast (default setting) and unicast. To use broadcast, please set Destination IP as 255.255.255.255. To use unicast, please set Destination IP as the same as PC IP address. 2) Users can choose the dual return type from the last return, the strongest return, and the dual return.

  • Page 26: Device Info

    4.3 Device Info Software version, hardware version, firmware version can be viewed from device information page. Figure 4.3 Device Info Page of Web Control - 2 3 -...

  • Page 27: Firmware Upgrade

    4.4 Firmware Upgrade Please ask Hesai for the latest upgrade file if needed. Click on “Upload” button to upload the upgrade file. Please restart Pandora after finishing upgrading. Figure 4.4 Upgrade Page of Web Control - 2 4 -...

  • Page 28: Lidar Channel Distribution

    Appendix I LiDAR Channel Distribution Channel Number of Laser Horizontal Angle Offset (Azimuth) Vertical Angle (Elevation) The horizontal angle of each line is the sum of The vertical angle of each line is constant, and 0° Channel number in UDP Data Packet current reference angle of the rotor and the angle represents horizontal direction.
  • Page 29 Channel Number of Laser Horizontal Angle Offset (Azimuth) Vertical Angle (Elevation) 0.00 -2.00 5.00 -2.33 -2.50 -2.67 0.00 -3.00 5.00 -3.33 -2.50 -3.67 0.00 -4.00 5.00 -4.33 -2.50 -4.67 2.50 -5.00 5.00 -5.33 -2.50 -5.67 2.50 -6.00 2.50 -7.00 2.50 -8.00 0.00 -9.00...

  • Page 30: Appendix Ii

    Point Cloud Data Packet Absolute Time and Laser Firing Time Calculations Pandora transmits two types of UDP Data Packet, including the Point Cloud UDP Data Packet and the GPS UDP Data Packet, hereafter referred to as Point Cloud Data Packet and GPS Data Packet.
  • Page 31 The UTC time in Pandora GPS Data Packet and following Point Cloud Data Packet are same and derived from previous GPRMC information. Although previous GPRMC is 1 full second older than the absolute time of the triggering PPS rising edge, Pandora has an adjustment mechanism so that the UTC time is exactly the absolute time of triggering PPS rising edge.
  • Page 32 Single Return Mode There are 10 Blocks in every Point Cloud UDP Data Packet. In the single return mode, each block consists 40 laser channels ranging data. The end time of the Block means all the 40 channels laser finished the firing. The calculation of each Block's end time is as follows: 01) End time of Block10: (t0-28.58) μs;...
  • Page 33 Through the Block's end time, it is possible to calculate the laser firing time for every channel in the Block. Take Block 6 for example, assuming Block 6's end time is t6, then: 1）Laser ID 4’s firing time: (t6-0.93*1-4.96) μs； 21）Laser ID 2’s firing time: (t6-0.93*12-1.6*9-4.96) μs；...

  • Page 34: Appendix Iii

    PandarView is a software that is used to play and record the point cloud data. Installations are available on platforms: Windows 7x64/ Windows 8x64/ Windows 10x64/ Ubuntu-16.04 The installation package can be downloaded from Hesai official webpage under “development data”. 1 PandarView Installation Please install the PandarView and set the computer static IP address to 192.168.
  • Page 35 2 PandarView Instructions Check Live Data Record Pcap Files In real-time play mode, click on the icon to pop up the Windows：Double-click shortcut and you will see an “Choose Output File” window. Click on “Save” to begin initial interface. recording a pcap file. Ubuntu-16.04: Double-click the shortcut on the desktop (if you set "Run executable text files when they are opened"), After recording, click on...
  • Page 36 Figure III.5 Ready to Play the Pcap File Import Correction File Each Pandora comes with a correction file (CSV) in the provided USB disk. In the play mode, click on “File” in the upper left corner. Choose “Import Correction File” in the drop-down menu, select and open the correction file to display the calibrated point cloud.
  • Page 37 Play Button Button Description Jump to the beginning of the file. 1. While paused, click to view point cloud data from the previous frame. 2. While playing, rewind (click again for different speeds, such as 2x, 3x, 1/2x, 1/4x, 1x speeds). 1.
  • Page 38 3 PandarView Features View Direction Selection 3D Projection Mode Click on the following buttons to view the point cloud from PandarView enables switching between two types of 3D projection different directions. methods (Orthogonal Projection and Perspective Projection) through the drop-down menu. In Orthogonal Projection view, click on , thereafter while holding “Control”...
  • Page 39 Point Cloud Data Selection Clicking on to select visible points. Users can hold down the Users can click on to display or hide point cloud data left button to box a certain area for selection. The selected from any selected laser channels. Clicking on this icon will points will be highlighted.
  • Page 40 Click to compensate the azimuth error caused by the delay of laser activation. Color Schemes By Clicking on , users can see the current color scheme in the Click on to open “Color Editor”, lower right corner. where users can customize colors. The drop-down list is used to choose different color schemes.
  • Page 41 PandarView Software Version Click “About” in the upper left corner to check the software version. Figure III.14 PandarView Softeware version - 3 8 -...

  • Page 42: Appendix Iv

    Warranty and Maintenance If any defect due to faulty software and/or hardware occurs within the warranty period, Hesai Photonics Technology Co., Ltd will provide free maintenance service. Some operations will violate the warranty, including but not limited to the following: 1) The purchase documents have been altered in any way, made illegible or lost.
  • Page 43 Legal Notice All texts, graphics, and pictures in this manual are subject to the copyright of Shanghai Hesai Photonics Technology Co., Ltd and are potentially protected by copyright through third parties. No part of the manual may be reproduced, processed, duplicated or published in any form by photocopying, reprinting or other process, without a written agreement.
  • Page 44 Hesai Photonics Technology Co., Ltd Phone: 021-80394947-876 Business Email: info@hesaitech.com Technical Support: 021-80394947-915 Service Email: service@hesaitech.com Website: www.hesaitech.com Address: Building L2, Hongqiao World Centre, Shanghai HESAI Wechat...

Table of Contents