Hesai Entity List: A Guide to Lidar Point Cloud Data

If you're working with Hesai Pandar series lidars, you've probably encountered the raw point cloud. Millions of tiny dots representing the world. But the real magic, the part that turns that cloud into actionable intelligence, often starts with something called the entity list. It's not a mystical document. Think of it as the sensor's first, best guess at organizing the chaos. It's the structured data output that categorizes those raw points into things—vehicles, pedestrians, lane markings. This guide cuts through the jargon to show you what the Hesai entity list actually contains, why it matters for your project, and how to use it without getting lost in the data.

What is the Hesai Entity List Really?

Let's be precise. The term "Hesai entity list" isn't an official product name from Hesai's website. You won't find a manual titled that. It's a community and developer shorthand for the processed, semantically labeled output you can derive from Hesai lidar data streams. Primarily, this refers to the data coming from their Pandar series sensors (like the Pandar64, PandarXT) when used with Hesai's own perception software stack or compatible third-party middleware.

The raw output is a point cloud: X, Y, Z coordinates, intensity, maybe a timestamp and laser ID. The entity list is the next layer. It's the result of running clustering and classification algorithms on that raw cloud. The sensor or its accompanying processing unit groups points into clusters and assigns each cluster a label: vehicle, pedestrian, cyclist, unknown.

Why This Data Structure is a Big Deal

You could process the raw point cloud yourself. So why care about a pre-generated entity list? Efficiency and focus.

For an autonomous vehicle compute stack, running raw point cloud segmentation in real-time is computationally expensive. If the lidar sensor itself (or a closely coupled processing unit) can offload that initial step, it frees up the main CPU/GPU for higher-level tasks like trajectory prediction, planning, and sensor fusion. The entity list provides a structured abstraction. Instead of "here are 300,000 points," it says "here are 12 cars, 3 people, and 20 lane boundary points." That's a language your planning and control modules understand much faster.

For mapping and survey work, the entity list helps in automatic filtering. Want to create a clean digital twin of a city block without cars? The entity list lets you quickly identify and remove points belonging to the vehicle class. It's a powerful pre-filter.

The Most Common Entities You'll Encounter

The specific classes can vary based on the firmware and perception software version. However, most implementations converge on a core set. Here’s a breakdown of what you're likely to see, based on common industry practice and data from Hesai's technical demonstrations.

But here's the thing: is the default list always enough? I once worked on a port automation project where shipping containers were the primary object. The default "vehicle" class didn't cut it. We had to work with the data pipeline to define a custom "container" entity, with attributes like ISO code and stacking state. The takeaway: understand the default ontology, but know you can often extend it.

How to Use This Data in Real Projects

For Autonomous Driving & Robotics

The entity list is feed for your object tracker. Instead of tracking raw points, you track entities. Their bounding boxes and velocities from frame to frame create trajectories. This is simpler and more stable. You also use it for scene understanding. A high density of pedestrian entities near a crosswalk triggers a more cautious driving policy. A cluster of unknown entities on the road ahead warrants immediate attention, possibly an emergency stop.

For High-Definition Mapping

You drive the lidar-equipped vehicle around. The raw data is huge. Using the entity list, you can automatically strip out all vehicle, pedestrian, and cyclist points, leaving mostly static environment points. This drastically reduces manual clean-up time. You can also extract specific features—like all pole or traffic sign entities—to build a landmark map for vehicle localization.

For Security and Perimeter Monitoring

Stationary lidar for site security. The system can be configured to only alert when an entity of type pedestrian or vehicle enters a virtual geo-fence, ignoring vegetation moving in the wind or small animals. This reduces false alarms by orders of magnitude.

Getting the Data: Parsing and Visualization

You typically access this data via Hesai's SDK or ROS drivers. It doesn't come as a simple CSV file called "entity_list.txt." It's embedded in the data packets.

Common output formats: Protobuf messages, custom binary structs over UDP, or as structured fields within ROS message types (like custom extensions to sensor_msgs/PointCloud2 or separate perception_msgs). The Hesai documentation for your specific sensor model is the first stop. Look for sections on "object output," "perception output," or "target list."

For visualization, tools like RViz (ROS) can display bounding boxes if the messages are formatted correctly. More advanced tools like Foxglove Studio or CARLA can also ingest and visualize these entities in context. The key is to ensure your data parser correctly interprets the byte structure defined in the SDK's header files.

A personal struggle I see often: developers try to parse the data without checking the endianness or the coordinate frame. Hesai data is often in a sensor-centric coordinate system (front is +X, left is +Y). Your world frame might be different. That mismatch causes entities to appear in the wrong place. Always, always verify the coordinate transform.