UE5 Lumen Implementation Analysis

Overview

Lumen is UE5’s GI system, it is different from the traditional real-time GI which only includes the contribution of indirect diffuse reflection. It also includes indirect diffuse reflection and indirect highlight, providing a new set of complete indirect lighting. Lumen supports both hardware-based RTX and software-based Trace algorithms. The starting point of this article is that Lumen GI uses the process, algorithm, and data structure analysis of indirect diffuse reflection part based on software Trace to understand the basic principle and operation mechanism of Lumen from a macro perspective.

 

The core of Lumen includes the following parts:

• The simplified expression of unique scenes. The scenes that provide GI reflection in Lumen are not composed of models but are composed of these model simplified proxy MeshCards.

• Based on the intersection acceleration of the distance field, the core acceleration structure used by the software Trace in Lumen is GlobalDistanceField and MeshDistanceField. The lighting expression is separated from the scene structure. There are two core components for buffering lighting information in Lumen: One is AtlasTexture that stores the highest precision lighting, each SubTexture corresponds to a MeshCard, and also corresponds to a small slice of the scene entity model; The other component is Voxel, which is generated during the runtime and is the main part of Lumen’s light information expression.

• GI solution based on Screen Space Probe, SSGI is integrated into the solution process, MeshCard AtlasTexture lighting solution based on MeshDistanceField Trace, Voxel lighting solution based on GlobalDistanceField Trace and other GI algorithms.

Data Structure

1. Intersection Acceleration Structure

There are two main Tracing intersection acceleration structures in Lumen: Distance Field and HZB in 3D space. Distance Field is divided into Mesh Distance Field (hereinafter referred to as MDF) and Global Distance Field (hereinafter referred to as GDF). MDF/GDF is used to accelerate the intersection of models (MeshCard/Voxel) in 3D space, and Hierarchy Z-Buffer (hereinafter referred to as HZB) is used to intersect SSGI’s RayCast in screen space.

2. MDF/GDF

Without BVH, RayTracing can only take uniform steps along the light direction with a fixed step length and intersect the scene after each step. In order to penetrate thin slices and small models without errors, the step size needs to be infinitely small. And for operational efficiency, it is hoped that the step length is as long as possible, and the two requirements are completely opposite. The diagram of the uniform step is as follows:

With MDF/GDF, this situation can be effectively changed, because MDF/GDF stores the position closest to the model surface, so with MDF/GDF, RayMarch’s step length in most cases can be greatly improved without penetrating the “model” error. As shown in the figure below, light is emitted from the front of the camera, the blue circle is the Distance Field (the shortest distance to the object) stored at the current point. Because there can be no other objects within this distance (circle), so every time the length of the step can be directly the distance of the Distance Field:

3. HZB

RayMatch in screen space and RayTracing in 3D space have completely similar stepping problems. The screen space lacks complete 3D scene information, so the hit judgment of RayMatch is determined by comparing the depth Z of the current pixel with the Z value of the light equation at the current position. The depth of the current pixel light is greater than or equal to the depth in the ZBuffer (the following discussion is based on the general ZBuffer, the reversed Z rule is the opposite), the light is considered intersected, otherwise, it is considered that it is unblocked, and you can continue to step forward. The basic idea of HZB is similar to BVH. It first generates a Mipmap chain for the current ZBuffer, where the higher Mip Z value takes the smallest value among the 4 pixels in the previous Mip line, and then selects one of them when executing RayMatch Level Mip starts. If there is no intersection in the current Mip, increase the Mip level and continue to intersect, otherwise reduce the Mip to intersect at a finer granularity. Its meaning is to see whether RayMatching has an intersection with the current block in the larger pixel block. If the intersection is found in the smaller-grained pixel block, and there is always an intersection, it will drop to the highest precision require的level to check whether the light intersects the current pixel: On the contrary, it will try to see whether there is an intersection point in the larger block, so as to quickly skip areas that do not require pixel-by-pixel steps. The RayMatch on the two-level HZB is shown in the following figure:

4. MeshCard and LumenScene

LumenScene is the scene used by Lumen in the calculation process. It is an incomplete simplified version of the real scene. For Trace-based algorithms, choosing a simplified scene expression means speeding up Trace, and at the same time, it also brings a loss of Trace accuracy. MeshCard is the basic component of LumenScene. In the sense of scene structure, it is the basic element of LumenScene. For LumenScene, there are no models in the scene – MeshCard is their substitute (Proxy).

 

The more common model simplifications in real-time GI are voxel (Voxel) and surface element (Surfel). The MeshCard in Lumen is of unequal width and height and only one coordinate axis faces Orientation ∈ [-x,+x, -y,+y,-z,+z] hexahedron. In terms of shape, it is a block-shaped building block:

 

LumenScene is built piece by piece with such building blocks, but it does not need to be built manually, it is automatically generated. The main function of MeshCard in Lumen is to provide the position and direction of the light sampling, which is used to greatly reduce the basic component of the light information buffer. The 6 orientations of MeshCard are the 6 basic functions of Ambient Cube1, even in the model there are probably as many as 6 MeshCards at the same location, and each MeshCard just expresses a different direction. This expression can have a higher fidelity to DiffuseLighting, and is widely used in various real-time and baked GI solutions of Probe-Based. At the same time, it has less amount of instructions when restoring data than SH.

5. Screen Space Probe, please feel free to read the full content on the en,uwa4d,com, there are lot of professional game tech articles.

Please feel free to leave your comments here. Many thanks for your support.