Hesai Pandora User manual

Pandora
All-in-One Sensing Solution
for Autonomous Driving
User’s Manual
HESAI Wechat
www.hesaitech.com
100-en-1801A2

Caution
To avoid violating the warranty and to minimize the chances of getting electrically shocked, please do not disassemble the device on your
own accord. The device must not be tampered with and must not be changed in anyway. There are no user-serviceable parts inside the
device. For repairs and maintenance inquiries, please contact an authorized Hesai Technologies service personnel.
Use of controls or adjustments or performance of
procedures other than those specified herein may result in
hazardous radiation exposure
CAUTION
The device satisfies the requirements of:
IEC 60825-1:2014;
21 CFR 1040.10 and 1040.11 except for deviations pursuant to Laser Notice No.50, dated June 24, 2007;
GB7247.1-2012
Laser Safety Notice-Laser Class 1
DISCLAIMER The information contained within this user’s manual and the functions offered are intended to provide information about
products. All reasonable efforts have been made to ensure the accuracy of the information. However, Hesai cannot be held responsible for
any errors. Hesai does not warrant the accuracy and reserves the right to make changes to the catalog and its functions at any time without
notice.
Please read and follow all instructions carefully and consult all relevant national and international safety regulations for your
application.
Safety Notice

Contents
1.1
1
Pandora-LiDAR 02-03
1.2 Pandora-Camera 04
1.3 Camera and LiDAR Synchronization and Calibration 05
1.4 Specifications 06
Introduction
2.1 Mechanical Installation 07
2.2 Interfaces 08
2.3 How to Connect 09-10
Installation
2
3.1 LiDAR Data Structure 11-18
3.2 Camera Data Structure 19-20
3.3 Analysis of Data 20
3.4 Pandora Projection for ROS 20
Pandora Data Structure
3
4.1 Open Web Control 21
4.2 Setting 22
4.3 Device Info 23
4.4 Firmware Upgrade 24
Web Control
4
25-26
Appendix I
LiDAR Channel Distribution
27-30
Appendix II
Point Cloud Data Packet Absolute Time and Laser
Firing Time Calculations
31-38
Appendix III
PandarView
39-40
Appendix IV
Support and Contact

Figure 1.1 Placement of Pandora
LiDAR
LiDAR has an extended
measurement range of 200m at
20% reflectivity, and generates up
to 720,000 data points per second
Cameras
Four wide-angle lens cameras and
one forward-facing color camera
capture 360° image around the car
Introduction1
Pandora is an all-in-one sensor kit for environment sensing for self-driving cars. It integrates cameras, LiDAR and data processing ability
into the same module, with advanced synchronization and calibration solutions.
In addition to the specifications of Pandora, this manual also describes the mechanical installation, data outputs format, and GPS timestamp
synchronization.
This manual is undergoing constant revision and improvement, please ask Hesai for the lastest version of the user’s manual.
-01-

The upside of Pandora is a 40-channel mechanical LiDAR. It creates 3D image by 360° rotating through 40 laser diodes inside the housing.
LiDAR’s unique channel distribution makes it more suitable for autonomous driving applications.
1.1 Pandora — LiDAR
1.1.1 Operational Principles
Distance Measurement: Time of Flight (ToF)
Figure 1.2 ToF Formula
d= ct
1
2
d：
c：
t：
Distance
Speed of light
Laser beam travel time
A laser diode emits a beam of ultrashort pulse laser on to the object.
Diffuse reflection of the laser occurs upon contact with the target object.
Reflected beams are detected by the optical sensor.
Distance to object can be accurately measured by calculating the time
between emission and receipt by the sensor.
1.
2.
3.
1.1.2 Structure Description
40 pairs of laser emitters and receivers are attached to a rotating motor
inside the Pandora that perform horizontal scans in 360 degrees.
Laser Receiver
Laser Emitter Shell
Interface
Figure 1.3 Partial Cross-Sectional Diagram
-02-

Channel 1
Channel 6
Channel 12
Channel 30
Channel 40
+ 7°
+ 2°
0°
- 6°
- 16°
Figure 1.4 LiDAR Channel Distribution
1.1.3 Channel Distribution
The vertical angular resolution is 0.33° between Channel 6 and Channel 30;
The vertical angular resolution is 1° between Channel 1 and Channel 6, Channel 30 and Channel 40.
Please see Appendix I for detailed channel distribution.
-03-

Camera
Black and White：129° Color：52°
LiDAR
360°
Pandora has four wide-angle lens cameras and one forward-facing color camera capturing 360° image around the car.
Figure 1.5 Coverage of Cameras
Figure 1.6 Camera Placement
No. Type Resolution FOV
0 Color Camera 1280*720 52° (H), 28.6° (V), 61° (D）
1 Mono Camera 1280*720 129° (H), 81.8° (V), 142.4° (D)
2 Mono Camera 1280*720 129° (H), 81.8° (V), 142.4° (D)
3 Mono Camera 1280*720 129° (H), 81.8° (V), 142.4° (D)
4 Mono Camera 1280*720 129° (H), 81.8° (V), 142.4° (D)
Table 1.1 Type of Camera
3
1
2 4
0 Color Camera 1-4 Mono Camera
Pandora—Camera1.2
-04-

Figure 1.7 Synchronization of LiDAR and Cameras Figure 1.8 Calibration of Multiple Sensors
Synchronization
Pandora controls the motor rotating and laser firing time of the
LiDAR, and at the same time, LiDAR controls the exposure time
and frame rate of the cameras. Therefore, Pandora can achieve
synchronization of point cloud data from LiDAR and imaging data
from the cameras, which makes sure those data are describing the
same object.
Exposure
Exposure
Calibration
Based on the calibration between point cloud and images, an
accurate mapping between point cloud and camera pixel can be
achieved and their spaces can be accurately matched.
Camera and LiDAR Synchronization and Calibration1.3
-05-

Specifications
1.4
LiDAR
Scanning Method Mechanical Rotating
Channel 40
Measurement Range 0.3 m~200 m（20% reflectivity）
Accuracy ±5 cm (0.3 m ~ 0.5 m),
±2 cm (0.5 m ~ 200 m)
Point Frequency 720 kHz
Laser Class Class 1 Eye Safety
Frame Rate 10 Hz, 20 Hz
FOV (Horizontal) 360°
Angular Resolution 0.2°(10 Hz), 0.4°(20 Hz),
FOV (Vertical) -16°~7°
Return Mode Dual Return
Angular Resolution (Vertical) 0.33°（-6° to +2°）；
1°（-16° to -6°，+2° to +7°）
Color Camera
Resolution 1280*720
FOV 52°(H), 28.6°(V), 61°(D)
EFL 5.47 mm
TV-Distortion ＜-14.2%
Mono Camera
Resolution
FOV
1280*720
129° (H), 81.8° (V), 142.4° (D)
EFL 1.65 mm
TV-Distortion ＜-44.2%
System Specs
Size Height: 190 mm, Top Diameter: 116 mm,
Bottom Diameter: 118 mm
Power Consumption 30 W
Data Transmission Ethernet (1000 Mbps)
Operating Temperature -10 ℃~60 ℃
Operating Voltage 9 V~32 V
Weight 2.1 kg
Enclosure Level IP66
Table 1.2 Pandora Specifications
-06-

Pandora has already finished calibration before delivery. The unit is easy to install and plug-and-play. Please refer to chapter 2 for details on
mechanical dimension, interfaces and connection methods.
NOTE Because of the intrinsic angle offset of each laser channel, the zero degree is defined as the azimuth angle in the corresponding block
in UDP packet when channel 12 passes y axis defined in figure 1.3.
Installation2
Mechanical Installation2.1
45°
φ118（0D）
2-φ4mm 5.5mm
FORφ4mm PINS
φ88.9 5-M6 5(MOUNT)
φ118
Figure 2.1 Mechanical Dimension Figure 2.2 Pandora Rotation Direction
270°
90°
180° 0°
Clockwise
Rotation
Direction
Reference Center
-07-

Interfaces2.2
Power Light
a b c d e
GPS port pin number from left to right is 1 to 6, and the specific
definition of each pin is shown as follows:
Table 2.2 GPS Pin No. DescriptionTable 2.1 Interface Description
Figure 2.3 Pandora Interfaces
Pin No.
1 Input PPS synchronizing signal, to receive synchronized
pulses from the GPS module, TTL 3.3V
2 Output 5V power, to provide power for external GPS module
3 Output GND, to ground external GPS module
4 Input Receiving signal of serial port, receiving serial data
from external GPS module, RS232 level
5 Output GND, to ground external GPS module
6 NC Not connected
Direction Pin Description
c Standard Ethernet Port
RJ45, 1000 Mbps Ethernet
d, e Reserved Interfaces
aGPS Port
Connector type: JST SM06B-SRSS-TB
Recommended connector for external GPS module: JST
SHR-06V-S-B
Voltage standard: RS232
Baud rate: 9600 bps
bbPower PortPower Port
Use DC-005 DC power adapter.
Input voltage ranges from 9 V to 32 V.
Power consumption is 30 W
-08-

GND, to ground external GPS module
How to Connect2.3
Connection of Pandora2.3.1
Pandora begins to scan and transmit data automatically once it is powered up and connected to the computer.
Web control can be used to set parameters of Pandora before using. For more on web control, see Chapter 4.
Users can use PandarView to quickly view or record point cloud data captured by Pandora. For more on PandarView installation and usage,
see Appendix III PandarView.
Pandora
Power Port and
Standard Ethernet Port
Connecting Box Computer
Connect the power port to
the adapter
Use an Ethernet cable to
connect the LiDAR’s and
computer’s Ethernet ports.
Figure 2.4 How to Connect
-09-

IP Configuration
2.3.2
To receive data on your PC, please set the PC IP address to 192.168.20.100.
Windows:
1) Open the Network Sharing Center, click on “Ethernet”.
2) In the “Ethernet Status” interface, click on “Properties” to proceed to the next interface.
3) Double-click on “Internet Protocol Version 4 (TCP/IPv4)”
4) Configure the IP address to 192.168.20.100 and subnet mask to 255.255.255.0, then click “OK” to finish configuration.
Ubuntu-16.04:
The IP address can be configured on the terminal by using the ifconfig command:
~$ sudo ifconfig enp0s20f0u2 192.168.20.100
Replace enp0s20f0u2 with the local network port name.
-10-

LiDAR Data Structure3.1
The communication protocol for data output of LiDAR is Gigabit Ethernet UDP/IP. The output data includes Point Cloud Data Packet and
GPS Data Packet. Each data packet consists of an Ethernet Header and UDP Data.
Figure 3.1 Data Structure of LiDAR
LIDAR Data
Point Cloud Data Pachet
Ethernet Header：42 bytes
UDP Data：1262 bytes
Ranging Data：1240 bytes
Additional Information：22 bytes
GPS Data Pachet
Ethernet Header：42 bytes
UDP Data：512 bytes
Users can receive both point cloud data from LiDAR and imaging data from cameras through Ethernet Cable. The data communication
protocol for LiDAR is UDP protocol, while that for camera is TCP protocol.
Pandora Data Structure3
-11-

Here is an example of the definition of Ethernet Header:
A Pandora has a unique MAC address.
The destination IP address is 0xFF and in broadcast form.
The default source IP address is 192.168.20.51.
Taking “Internet Protocol (20 bytes)” as an example, it is described as follows:
Ethernet Header: 42 bytes
Table 3.1 Definition of Point Cloud Data Packet Ethernet Header
Ethernet II MAC
Ethernet Data Packet Type
Internet Protocol
UDP Port Number
UDP Length and Checksum
12 bytes
2 bytes
20 bytes
4 bytes
4 bytes
Destination: Broadcast (0xFF: 0xFF: 0xFF: 0xFF: 0xFF: 0xFF), Source: (xx:xx:xx:xx:xx:xx)
0x08, 0x00
Version, Header Length, Differentiated Services Field, Total Length, Identification, Flags, Fragment Offset,
Time to Live, Protocol, Header Checksum, Source IP Address, Destination IP Address
UDP source port (0x2710, represents10000), destination port (0x0940, represents 2368)
Length 2 bytes (0x04F6, represents 1270 bytes), checksum 2 bytes
Figure 3.2 Point Cloud Data Packet Ethernet Header Internet Protocol
Point Cloud Data Packet—— Ethernet Header
3.1.1 Point Cloud Data Packet
Each Point Cloud Data Packet has a 42 bytes Ethernet Header and 1262 bytes UDP Data.
-12-

The UDP Data of Pandora has a 1262 bytes payload consisting of 1240 bytes ranging data and 22 bytes additional information.
All the multi-byte values are the unsigned type and in Little Endian format.
The definition of each block in ranging data is as follows:
NOTE Under dual return mode, azimuth angle changes every two blocks. The odd number block is the last return, and the even number block
is the strongest return.
Table 3.3 Definition of Each Block
Table 3.2 Point Could Data UDP Data-Ranging Data
Ranging Data 1240 bytes (10 blocks)
Block 1 Block 2 Block 3 Block 10
0xFFEE 0xFFEE 0xFFEE 0xFFEE
Azimuth Angle 1 Azimuth Angle 2 Azimuth Angle 3 Azimuth Angle 10
Channel 1 Unit 1 Channel 1 Unit 2 Channel 1 Unit 3 Channel 1 Unit 10
Channel 2 Unit 1 Channel 2 Unit 2 Channel 2 Unit 3 Channel 2 Unit 10
Channel 40 Unit 1 Channel 40 Unit 2 Channel 40 Unit 3 Channel 40 Unit 10
······ ······ ······ ······
······
······
······
······
······
······
······
Each Block in Ranging
Data: 124 bytes
0xFFEE 2 bytes Head, meaningless, 0xFF first
Azimuth Angle 1 2 bytes
Represents the current reference angle of the rotor.
Azimuth [15:0]: lower byte Azimuth_L [7:0] is in the front,
upper byte Azimuth_H [15:8] is in the back.
Azimuth Angle=[Azimuth_H, Azimuth_L]/100°=Azimuth/100°
Channel XX Unit XX 3 bytes 2 bytes distance data Distance Value = Distance*4mm
Maximum Distance Value = (2^16 – 1)*4mm
= 262.14m
1 byte reflectivity data
Point Cloud Data——UDP Data
-13-

Additional Information: 22 bytes
Table 3.4 Point Cloud Data UDP Data-Additional Information
Reserved 8 bytes reserved data, meaningless
Motor Speed 2 bytes speed_2_bytes [15:0] = speed (RPM)
GPS Timestamp 4 bytes the packing time of this data packet, the unit is 1 μs, value range 0 μs-1 s
Return Mode Information 1 byte the strongest return (0x37), the last return (0x38), dual return (0x39)
Factory Information 1 byte 0x42 (or 0x43)
UTC 6 bytes year, month, date, hour, minute, second, decimal digit
Taking Channel 5 in block 3 of a UDP Data Packet as an example, please see Appendix I for detailed channel distribution:
1）
2）
3）
By now, the direction and distance of this point have been decided, and this obstacle point could be drawn in the polar or rectangular
coordinate system. The real-time point cloud data can be drawn by analyzing every data in the UDP Data Packet using the above method.
Horizontal angle offset of the laser is -2.50°, and vertical angle of the laser is 3.00° for Channel 5.
Horizontal angle is the current reference angle of the rotor plus horizontal angle offset, so the result is (Azimuth Angle 3+(-2.50)) degree.
(NOTE We define clockwise as a positive direction of the angle from top view)
Analyze the “Channel 5 Unit 3” from the UDP Data Packet, and the distance formed by upper 2 bytes multiplied by 4mm is the actual
distance in millimeters in the real world.
Example of UDP Data Analysis
-14-

3.1.2 GPS Data Packet Ethernet Header/ UDP Data
Each GPS Data Packet has a 42 bytes Ethernet Header and 512 bytes UDP Data, and the port is 10110.
Before receiving the GPS module data, the rising edge of the internal 1Hz signal of Pandora will trigger a GPS Data Packet. GPS time data
will be counted from 000101000000 (yymmddhhmmss, year, month, day, hour, minute, second) and GPRMC information is not available at
this time.
After the GPS module data has been sent to Pandora, the local 1 Hz signal of Pandora will be locked at PPS signal. The rising edge of the
internal 1Hz signal will trigger a GPS Data Packet. Meantime, the GPS time data will be reset to actual GPS time and GRPMC information will
be the original data sent by GPS module.
If GPS module stops sending data, Pandora will still trigger a GPS Data Packet following the internal 1Hz signal. GPS time data will be
counted on the base of previously actual GPS time.
Ethernet Header: 42 bytes
Table 3.5 Definition of GPS Data Packet Ethernet Header
Ethernet II MAC 12 bytes
2 bytes
20 bytes
4 bytes
4 bytes
Destination: Broadcast (0xFF: 0xFF: 0xFF: 0xFF: 0xFF: 0xFF), Source: (xx:xx:xx:xx:xx:xx)
0x08, 0x00
Version, Header Length, Differentiated Services, Field, Total Length, Identification, Flags, Fragment
Offset, Time to Live, Protocol, Header Checksum, Source IP Address, Destination IP Address
UDP source port (0x2010, represents 10000), destination port (0x277E, represents 10110)
Length 2 bytes (0x208, represents 520 bytes), checksum 2 bytes
Ethernet Data Packet Type
Internet Protocol
UDP Port Number
UDP Length and Checksum
GPS Data Packet – Ethernet Header
Here is the definition of GPS Data Packet Ethernet Header:
-15-

The destination IP address is 0xFF and in broadcast form.
The default source IP address is 192.168.20.51. Taking “Internet Protocol (20 bytes)” as an example, it is described as follows:
GPS Data Packet - UDP Data
Figure 3.3 GPS Data Packet Ethernet Header Internet Protocol Illustration
UDP Data of GPS Data Packet consists of 18 bytes time data and 494 bytes additional information. All the multi-byte values are the
unsigned type and in Little Endian format.
Table 3.6 GPS Data Packet-UDP Data Definition
GPS UDP data: 512 bytes
GPS Time Data 18 bytes
GPRMC Data 77 bytes ASCII code, valid till 2 bytes after ‘*’
Filled with 411 0xDF
From GPRMC information, ASCII code, A=valid, V=invalid
1=locked, 0=unlocked
Reserved meaningless data
Reserved Data 411 bytes
Location valid or not 1 byte
Flag of PPS lock 1 byte
Reserved Data 4 bytes
Header
Date
Time
μs Time
2 bytes
6 bytes
6 bytes
4 bytes
0xFFEE
Year month and day in order (2 bytes each), lower byte first, ASCII code
Second minute and hour in order (2 bytes each), lower byte first, ASCII code
Unit is μs, lower byte first
-16-

Date：
Year: 0x37, 0x31, convert ASCII code to '7', '1'; means 17
Month: 0x32, 0x31, convert ASCII code to '2', '1'; means 12
Day: 0x30, 0x32, convert ASCII code to '0', '2'; means 20
Figure 3.4 GPS Data Packet UDP Data Illustration
Time：
Second: 0x32, 0x35, convert ASCII code to '2', '5'; means 52
Minutes: 0x35 0x34 convert ASCII code to '5’, '4; means 45
Hour: 0x32 0x31, convert ASCII code to '2', '1'; means 12 (UTC time)
μs Time：
4 bytes, the μs time value of each GPS PPS pulse, and timestamp will be set as 0 μs.
The μs time of GPS PPS and the timestamp from the point cloud data have the same data source, and the unit is 1 μs.
Example of GPS Data Packet UDP Data Analysis
-17-