Lighting Studies in UE4: Part 2

Adam Alexander gave another interesting talk on the lighting production in UE4 and broke down the projects he’s recently worked on. In his experiments, Adam continues the studies of lighting after passing CGMA course The Art of Lighting for Games.

What I’ve Been Up To

Hi! Since I last wrote about lighting with CGMA’s “The Art of Lighting for Games”, I’ve been working on some personal scenes trying to focus on lighting and composition.

Part of this has been studying the work of a number of photographers that I really admire and trying to create scenes in Unreal 4 inspired by some of their photos. These photographers do an excellent job of creating mood and atmosphere in their shots, and it is something I want to try to capture in my own scenes. For me, these photos make me want to know more about the people or places depicted and to piece together whatever story may be hinted at in the images. These are critical concepts for creating art for games, and there is a lot to be learned by studying other mediums!

Night Scene Study

For my most recent studies, I based my scenes off photos in Jennifer Garza-Cuen’s “Reno” series and Todd Hido’s “Homes at Night” series. My Night Scene Study tried to re-create some of the elements I found compelling in Todd Hido’s photos. In these beautiful yet slightly foreboding shots, dense fog, hazy lights, and silhouetted shapes transform rural scenes into surreal, dreamlike locations. I knew from an artistic standpoint, I wanted to try and capture those qualities in my own study of his work, with a strong focus on creating a particular mood. From a technical side, I wanted to develop a better workflow with Unreal 4’s Volumetric Fog system, and to try to use it to get a particular look that mimicked some of the light shapes in Hido’s pictures. Additionally, I knew I wanted to create a small video of this scene as an excuse to explore adding some motion into it: rolling fog, blowing leaves, a little bit of camera motion, blinking tail lights, etc. This was a perfect opportunity to become more familiar with Unreal’s Sequencer tool!

Pre-Made Assets

I knew that since I wanted this scene to serve as a study for scene composition and lighting, I did not want to spend a ton of time creating all the individual, high-detail assets that would go into it. For situations like this, there is a great number of materials and meshes available that can be quickly edited to suit your needs! I will also add that, while this is a great way to focus on certain elements of a study, it is important to point out what elements in your scene are your work, and what is the work of others (especially if you plan on using this as any sort of portfolio piece!) Additionally, give credit to the original artists who produced the content you are using.

For the Night Scene Study, I already had a pretty big library of some personal assets and materials I had made from previous projects, but I grabbed and edited a couple of other assets to make my scene. I used some particle effects and textures from Epic samples to create some of my own fog particles and materials. For the foliage and root assets, I used some of the wonderful content Unity released for their Book of the Dead film. Additionally, I used some of Clinton Crumpler’s asphalt textures from his Laundromat scene for my road blend material and Unity store user rfarencibia’s pick up truck model. This all helped me focus my attention on the layout of the scene and the lighting!

Adding Motion Into The Scene

Creating a video of my scene allowed me to introduce some moving elements into the composition. I layered in some blowing fog particles to add a little movement on top of the volumetric fog. I also used a simple vertex animation for my foliage assets to get them to flutter in the wind. My favorite motion element I added was the blinking tail lights and exhaust from the truck: from a storytelling perspective, I felt it made the scene more ominous. Who had abandoned their vehicle idling by the side of the road in the middle of the night? The simple effect was created by having two redpoint lights’ intensity tied to a looping timeline that was triggered by an event in the Level Sequence. I drove the emissive value of the taillight material instance with the same variable.

Volumetric Fog

Unreal 4’s volumetric fog system has been a really great addition for lighting! Now, dynamic lights are represented in the volumetric height fog actor, and as of the 4.18 release, even static lighting can affect volumetric fog via Unreal’s new volumetric lightmaps. For my Night Scene Study, I leaned heavily on volumetric fog for both creating the main light shapes in my composition and for adding depth to a dim, diffusely-lit scenario.

The volumetric fog can look pretty good without much tweaking, but I did have to find some unique solutions for getting the look I wanted. For example, the user doesn’t have much control over the specific shape of the volumetric light; it will match the intensity, falloff, and attenuation of your light source. Initially, I knew I wanted to use a spotlight as the main type of light for the streetlamp, but I found that the volumetric light cone terminated at an unnaturally sharp point at the top of the cone. I was able to offset this effect by adding a second, smaller point light near my light source to diffuse the shape of my spot light’s cone. Building lighting and allowing the light probes to pick up some of the bounced lighting from the light source also helped naturally diffuse some of the shape! In general, I find that the volumetric fog is a great tool, but I still need to rely sometimes on old-school techniques of using meshes or cards to represent god rays and specific volumetric shapes.

Optimizing for Performance

Creating this scene, I was mostly concerned with getting the look I wanted and capturing a rendered image and a small video. I didn’t worry about performance too much, and I allowed myself to use as many of Unreal’s bells and whistles as I wanted. But you can achieve a very similar look at a much cheaper cost with a couple of changes! I’ll outline some steps I could have taken if I wanted to take something like Night Scene Study and optimize something like a game environment. Unreal provides a very useful tool for analyzing GPU expenses called GPUProfiler. You can access it and take a snapshot of your scene by opening the console command window with the tilde key and entering “gpuprofile.” This will let you pick apart the various parts of your scene and see which elements are costing what.

To begin, it is important to understand the different types of lights in Unreal and how expensive they are. Unreal splits it’s lights into three different categories: static, stationary, and movable. Static lights are very cheap since all their work is pre-calculated. However, they will only show up as static lightmaps and in volumetric lightmaps, and you lose a lot of what dynamic lights give you: specular read on materials, representation in volumetric fog, etc (Since I was doing a foggy, nighttime scene, these two things were very important!). Stationary lights are a bit cheaper; their direct lighting is dynamic, but they will also bounce light like a static light, which will be represented in the volumetric lightmaps and surface lightmaps. Movable lights are fully dynamic and the most expensive; they don’t get calculated in light builds at all. For my scene, I used mostly stationary lights with a couple of movable lights for things like the tail lights.

The most expensive part of dynamic lights in Unreal 4 is dynamic shadows. If I wanted to begin to optimize my scene, I would go through and evaluate what each dynamic light in my scene was doing. A lot of the time, a light can be set to not cast a shadow with very little impact in look in your scene but with a huge performance gain! I went through and disabled shadow casting on some secondary lights. I disabled or converted a couple of detail lights that were not crucial to the scene to static. Additionally, I scaled in the distance my light source’s cascaded shadow maps extended, and switched my shadow casting dynamic lights to use raytraced shadows from mesh distance fields. I also dropped my overall dynamic shadow resolution. In the end, I brought down my shadow cost from 7.83ms to 1.96ms with a couple minutes of work and only a small drop in quality!

As I began to go through other expensive elements in my scene, I realized the planar reflection actor was costing me a whopping 15ms. While this added the nice touch of accurate reflections in the puddles in my scene, it was an easy choice to cut this and rely on static reflection capture actors and SSR for the optimized version of this scene. Cutting the quality of the volumetric fog was another easy choice, it dropped from around 2.3ms to 0.37ms.

In the end, my changes meant the whole scene’s cost per frame went from around 53ms down to about 20ms. There are still a lot of areas I could continue to improve; the material blends on the ground and road are still pretty expensive, and the fog cards and particles create a lot of overdraw, especially with all the foliage in the scene. Creating particle LODs and streamlining my material graph would be a good logical step in continuing to optimize this scene. You can see the before and after versions of my scene below.

Adam Alexander, Lighting Artist

Interview conducted by Kirill Tokarev