General Pipeline and Lighting Study: Video Games vs. VFX Commercial

About the Artist

Known as a highly prolific and creative CG Artist, Xiaoya Zhao has made significant contributions to internationally renowned and award-winning films, commercials, and video games. As a VFX Artist for film, her work includes Spring Flower (Nominee, Best Short, Montreal World Film Festival, 2018) and Free Choice (Winner, Audience Award, Hollyshorts Film Festival, 2018). An essential CG artist for commercials and brand campaigns, Zhao’s highlights include a number of collaborations as a 3D Artist for global creative agencies The Mill for clients including Audi, Nike, and Nintendo and Moving Picture Company (MPC) for clients including PlayStation, Fiji Water, and Starburst.

She brings to each role advanced proficiencies in a number of skills and leading tools: Modeling (Maya, ZBrush), Texture (Mari, Substance Painter, Substance Designer), Look Development (Arnold, V-Ray, Mantra), and Lighting (Maya, Houdini, UE4) to name a few. As a result, Zhao is constantly sought after for her expertise as a CG Artist by some of the industry’s leading directors and artists. Her contributions have been covered by the entertainment and commercial industries’ leading publications: Animation World Network, Ad Age, AdWeek, MediaPost, Fast Company, Tech Radar, and New York Business Journal.

Pipeline Comparison

From the two images below, you can easily see the difference between video game and VFX commercial pipelines. Most of the CG work for a video game is being done during the production phase. The pre-production mainly focuses on the story and concept. Once the concept and the design of the gameplay approved, the whole team will move on to the production phase. Most of the companies will start hiring more CG artists at this moment since they will need a lot of CG artists to work on the environment/character modeling, texture, look development, animation, and lighting. All the assets are pushed from the block out phase to the proxy phase and then to the beta phase. We will move on to the content bug fix after the beta phase. 

Most of the VFX production is starting at the post-production phase of the commercial project, that’s why we also call VFX studios post-production companies. Big VFX companies usually have their own design department, so they can start some of the pre-production work in house from the very beginning. But it depends on the project, some of the clients prefer to have the pre-production being done in agency companies since they are more focused on design.

The VFX commercial pipeline image I made explains the pipeline for photo-real VFX commercial projects. Compared to a full CG project, we will need to spend more time on the onset shooting. In order to fully match the lighting of the onset environment, we also need to take HDR (high dynamic range) images and material reference images following certain rules while we are doing onset shooting. For a full CG project, we usually start VFX production once we have the story and concept from the pre-production phase. 

Lighting Workflow Comparison

Lighting process in VFX commercial  

1. Organize, stitch and calibrate the onset shooting HDR images. This step will make sure we have the HDRI from each onset shooting location ready for the light rig. We also need to name it properly, the naming should include timing and location.

2. Build light rigs for each environment/shot. Since we have the stitched and calibrated HDRI ready, we can start creating shots, add a tracked camera, backplate to each shot, create a CG grey and chrome balls, add direct light and reflection planes to make sure we can match the CG balls with the one on the plate. 

3. Render the first pass lighting with animation and pre-comp the render with backplate (shooting footage) for each shot. In this step, we need to make sure we have the render setting set as production level. We add animation sequence for each shot and set up render layers to make sure it can help us on light comp. 

4. Render the second pass with director/supervisor's notes addressed.

5. Render the final pass with all director's final notes addressed, hand on the render and tech passes to the compositing department, support them to finish the final look of the project, and make sure the project can be delivered on time.

Lighting process in Video Games

Compared to photo-real commercial lighting, video game lighting will give the artists more opportunities to design the lighting/the mood of the environment. But we need to follow the concept or the main look of the game first. Since we have more space to design the environment with lighting, it will require lighting artists to communicate with directors or concept artists more. 

The following steps are common for the video game lighting process:

1. Block out lighting pass: just basic support for gameplay. Make sure players can see all the assets in the environment. No dark areas. You don't need to perfectly match the mood of the concept. 

2. Proxy lighting pass: start matching and designing the look of the environment lighting, bake reflection maps, add indirect bounce. 

3. Beta lighting pass (final lighting pass): fully address the notes from directors on lighting, check lighting performance, make sure lighting is under the budget, and keep testing the game in modules. 

4. Lighting bug fix: keep working on the lighting bugs assigned by directors or gameplay artists.

These four phases of game lighting usually depend on the progress of the environment team.

Challenges of Lighting in Video Games

The first challenge I had was to understand how to control in-game lighting performance because most of my past experiences involved using pre-render software solutions. The lighting and rendering performance in pre-render software is mainly decided by the render setting, which is way much easier to control.  But for video game lighting performance, we have more aspects that need to be considered. For instance, how many dynamic shadow casting lights are in the level, whether or not the dynamic lights’ camera fade distance setting is accurate, the amount and resolution of the reflection map, how dense the light probes (indirect lighting) are, how many lights are overlapping, etc.

The second challenge is to work on lighting “without camera”.  For VFX projects I’ve worked on, we needed to focus only on making sure the render from the camera view is looking good enough. While working on video game lighting, we need to make sure the entire game environment is properly lit, which means we will need to spend more time tweaking the lights from different angles. 

Afterword

I feel so lucky to have the opportunity to work on many great projects both from the VFX industry and the video game industry. It was a lot of fun and a big challenge. I would like to thank all the people who gave me feedback and suggestions.  

Xiaoya Zhao, CG & VFX Artist