Secrets of Human Shaders in UE4

Incredibly happy to showcase a detailed breakdown from Dzung Phung Dinh. In this blog post, he provides full breakdown of his amazing Exparia project.

Introduction

My name is Dzung Phung Dinh. I got the 1st place winner in modeling demo reel contest of the very first year of Autodesk Contest for students ( CGSA ) back in 2010. Now they call it “The Rookies”.  I have been working in 3D graphic industry for nearly 10 years. In my early years, I worked as Game Artist for Rare UK and worked on a series of marketplace assets for some Xbox games: Fable III and Halo Reach, Later on I moved on to VFX/film industry, working as Creature Effect & generalist artist for Double Negative London on feature films like John Carter Of Mars, Snow White & the Huntsman, 007: Skyfall.

But these are things of the past, I am now the Founder and Team Lead of my own studio – Thunder Cloud, based in Vietnam, which creates exclusive quality assets for worldwide game development studios, which need support or which are outsourcing their art production. Our ultimate goal at Thunder Cloud is to become an animation studio creating animation & cinematics, using real-time tech. That’s why at the present, UE4 is one of my focus and key R&D projects.

EXPARIA PROJECT

As you already might know that back in 2016, ArtStation held its official contest “The Journey”.

I was very inspired by the theme and got fired up. I joined the contest and planned to create an epic 3D scene with the theme. The total duration of the contest was around 3 months. I think that’s a fair amount of time for an artist to be able to come up with something good, if he works full-time on it. Unfortunately, I got caught up with various jobs from Thunder Cloud and I didn’t finish my entry by the time the contest has ended. I didn’t really have enough time or maybe I was too ambitious with the piece. But recently I decided to pick up the project and finish it since it’s totally my personal project now, I wanted to be a little ambitious again.

My goal is to create an epic cinematic scene with the theme of “The Journey”. The main character is a girl named Exparia. There’s a wordplay of – Explorer (in my gallery all of my girl characters have names of something like ”-aria”). She is a Mech Pilot on a mission to explore a new world for mankind. There is a little story behind this scene, as you can see in the concept, a giant wormhole appears above the Earth orbit, some kind of stone alien structures are floating around it, which tells us that this wormhole is no work of nature. It was built artificially and has some sinister purpose. Mankind decides to send a voyage fleet to explore what lies behind the wormhole. All the mechs carried in the fleet are called “Kamikaze”, which means they are the first to go to the front line but most likely they won’t come back. They are trained as both scientists and soldiers at the same time. And our main character – Exparia is in one of the mech swith a smile on her face – showing her urge to explore the new world, anxious about the potential threats she might face.

The final result of the project will not be just a static image but also a small cinematic shot with animation & effects – all done in UE4. By working on this project I really want to push my skills in terms of both art and technique. There are organic characters with clothes, robots/mechs, hard surface indoor environment and open world environments in the background, etc. There are a lot of things that I can push and challenge myself to refine my skills, knowledge in the workflow and different tools to make it work best in UE4. And like I said above, all workflow is resolved around UE4.

One small note is that, since there are so many things listed down, you might ask what’s the main element in this project? It’s the main character & UE4 potential ability in rendering quality images & animation for production. As my background is more of character art and Thunder Cloud’s strongest service is character art. So the project best serves as a R&D and training project for me and the team at Thunder Cloud.

2D Concept

A good 3D artist is a role that stands somewhere in between, blending artistic & technical 3D stuff. I have been training painting digital 2D art and concept at the same time I picked up 3D. Being able to work as a 2d artist allows me to think much more freely in terms of concept, design & color, not being limited to what’s available in 3D tech and software. In this particular project, I  aim toward a very stylized visual style of a digital 2D painting with a little bit of PBR shaders look.

Here is the 2D piece I painted earlier, that I used as the main concept for the girl. I used this piece as the main reference to develop my shaders & textures. I have to break down each element in the 2D painting to make it look like a painting and attract the viewer and at the same time, using my skills in 3D, to translate it into 3D techniques. I also learn and look at works of other 2D artists, how they “render”/interpret and solve certain areas of a 2D character.

For example, the eye in 2D digital painting style feels much more natural and soulful to me than that of a 3D model generated and computed by 3D renderer in my opinion. First, I train myself to be able to paint in 2D, learn the fundamental structure of a certain 2D style. Then I start to mimic all of that structure and stylized elements in 3D and UE4 in this case.

Regarding the overall project, production-wise, to be able to draft out a 2D concept first allows me to predict how the final look of my 3D scene gonna look like in the end. It sets a standard of visual quality that my 3D model needs to achieve. It also helps me to determine the importance and priority of certain assets in the scene and come up with appropriate workflow and time needed to be spent on it. For instance, looking at the initial concept, of course. the girl and her Mech, would be the key element but the hangar floor around her also needs a certain level of detail. There are giant spaceships and vast open landscapes in the background, but they are very far away and dimmed by environmental fog so I can determine an appropriate level of detail to them as well as the best workflow to work on them whether they need high poly normal map or just pure model and diffuse, or even a matte painting could work in case of the far away mountain landscape, etc.

Pipeline

We are providing our character production services to many clients all over the world with vast variations of style and specification requirements from mobile game to high end.

We have one kind of a single general “mother” pipeline for all characters of all types, this might sound ridiculous to you but first we apply this universal pipeline to a project “on paper” and then strip down the unnecessary part or pick up a suitable workflow for some certain part then in the end we have the best optimized pipeline that’s specialized for a project while maintaining the main workflow thus everyone in the team picks up the work really quick not being confused by having different pipelines for different projects as they are all derived from a single “mother” pipeline.

Here you can see this is what a “mother” pipeline looks like.

This is an example of a strip down version of the pipeline for a low poly hand painted texture character.

Below is a more “completed” version which I used in this project that’s suitable for stylized PBR characters that requires high poly sculpt version and baking (Overwatch would be a good example)

Tools & Software

As for the tools I and team are using at Thunder Cloud, for different steps in the pipeline I used variety of tools from common ones like Maya as my primary modeling tool, Zbrush as primary sculpting tool, xNormal for baking, Substance Painter for PBR masking to uncommon things like BodyPaint 3D for texturing, Crazybump for map conversion and then a couple of Thunder Cloud in-house tools I wrote myself to aid with the character production. As you might already know – the hair tool GMH2 is my creation and I am using it to convert my hair base meshes into hair cards & textures for a character of this project. More information of GMH2 can be found here.

WIP of Hair model I am working on using GMH2.6

There a lot more in-house tools wrote by myself to aid with the production of this project such as:

Baking manager to support and speed up the process of baking from high poly to low poly a little bit less hectic and stressful, it helps you do excluded baking, mesh auto separation for ID baking, cage auto generation etc. I am looking into a way to let it communicate with xNormal to do batch baking.

Export Manager: a little script to help export a character with its material name really quick, which allows me to switch between UE4 and Maya in a really smooth and easy way.

Art Direction & Production

In this particular project, I am aiming at a very clean stylized PBR 3D look similar to Overwatch, Blade & Soul etc. or 2D digital paintings which can be often seen in works by artists like: Mujiha, BdsPlace. Thus I don’t really invest my time into detail sculpting, but rather keep everything neat and clean, I pay attention to mostly big & medium details like big bevels & edges, big clothes folds, an overall shape of the characters’ characteristic like nose, eyes, lips etc.

The small detail in sculpting would not be much visible after heavy paint over in texturing part and they are difficult to change and maintain. So for high poly sculpt model, I only stop at medium & big details that are visible in baked out maps later.

As for low poly model, like many high-end next gen games nowadays, I am planning the main character tris count around 50k tris and another 40~60 tris for the hair, so I have plenty of polygons to build my low poly model. I model every object as a separate mesh for easier baking and, of course, they are used for cinematic scene after all, so they shouldn’t be too low poly but still light-weight enough to run smoothly in UE4.

Hand-painted texture plays a big role in my production which influenced the quality of the look & feel of the model. I don’t usually paint these textures from a scratch. I baked out a series of maps from Xnormal, then combined & filtered out these maps in Photoshop to quickly create a base map that represents the final result I want, as close as possible.

This way I am able to preview and control the final result of the texture easily and quickly. I call this base texture “Procedure map” as it’s entirely made in Photoshop using filters and some simple masking, no painting involved at all. Once I am all happy with the look, color tone and overall shape/volume blocked out by the Procedure map, I start drafting out the texture by painting over it in BodyPaint. The reason I choose Bodypaint is that with a little tweaking to its UI and hotkey, it behaves a lot more similar to Photoshop than any other texturing software, so I feel very comfortable working in Bodypaint and I can easily switch back & forth between BodyPaint and Photoshop while working on the same PSD file.

In term of stylized character, to me the diffuse texture is the most important map to get right before everything else. Once I have a decent diffuse texture of everything, I will bring the model straight into UE4 for further look dev. In UE4 I will have to set up an environment with proper lighting and HDRI to troubleshoot and preview other maps such as roughness, specular, normal.

So far my workflow seems pretty linear, but from this point when I finish painting the diffuse map it’s pretty much just abck & forth between modeling, tweaking diffuse, normal and all other maps and developing the shaders bit by bit until I have a result.

Materials

Actually, I’ve just picked up UE4 recently, so I am still having a lot of things to learn. I don’t make super complex material functions or write my own custom shader nodes as I am not really that of a tech guy. So everything I am having now is pretty straightforward UE4 basic nodes & graphs.

I start with a base material in UE4 that I feel closest to what I want, for example, the skin material. I begin with a Subsurface shader, then start working my way up from there, looking at the 2D painting and watching, which element is missing and find a way to replicate the same effect in UE4. Here is a step-by-step breakdown:

• Base Material ( SubSurface shader) with basic parameters as a starting base.

Diffuse Color

• Input Base Texture.
• Add a fresnel mask with darkened skin texture to control the effect of gradient from silhouette similar to the way 2D illustrator often do their shading. ( look at Street Fighter for example in 3D)

This effect can be adjusted using strength and falloff for different result.

Multiply diffuse texture with a green color and plug in Subsurface Color slot to see the subsurface effect

Create a Fresnel mask to blend 2 subsurface colors together to mimic the subtle color shifting gradient shading in painting.

I create a separate material function fresnel mask that has control over gradient position and strength. This function also allows to add some noise to custom fresnel mask for a more organic skin look. Below is a closer look at the gradient function:

Using different strengths, falloff values and extra blended textures like vein & spore for more organic look. Different combinations can generate various looks and potentially be applied to other things aside from Skin shader.

Roughness/ Specular & Custom Reflection

Some reflection/hi-light areas in 2D painting have many hot spots and small sparkles, almost like a car paint material, so I add some micro detail noise map on top of my base roughness / specular map /normal map by overlay blend with a micro grunge texture to add small details to specular when viewing up close.

However, I still need more hi-light spot in the reflective area. I find a setup to control the reflection/highlight spots without having to mess up with scene lighting too much.

This one is a bit tricky, I set up this graph mainly solving around the Reflection Vector node of UE4. Plug it into UVs of a texture node we will have some kind of spherical environment reflection texture.

However, I did several tests and found out that it needs Cube map as a texture sample for the node to work correctly. The process of converting a 2D image from Photoshop into Cube map is fairly simple but still a very manual task and could be very time-consuming if we had to do it back & forth many times to test the result. So I use a flat long environment reflection 2D texture fromPhotoshop and apply it to an unlit sphere, then place a render target cube inside the sphere to create a cube map render from inside the sphere. Doing it this way, I can update my reflection map in Photoshop, reimport and update my cube map in UE4 instantly. Screen blend this reflection map with the color texture along with fresnel mask to have a decent custom reflection blended into our color texture.

I also made a custom material function to control some parameters of the reflection map, such as the strength of the hi-light area, base reflection value, fresnel falloff, specular map for masking, etc.

These are the overall setups of the skin shader, I haven’t tested it outside of this project yet but I am pretty happy with the final result that I can maintain the 2D/ hand painted look while still managing to retain the lighting information or even being able to add extra layer of detail onto its like reflection & subsurface color. That ‘s pretty much everything for the skin shader, later I will talk about the setup for the eyes which is more complicated but more or less based on the same principal.

Goal & Limitations

My ultimate goal is to create a 3D cinematic animation that looks like a digital 2D painting. So this sort of project was to push my limit of what can be achieved visually in UE4. Thus as long as the result looks good enough and strong enough to behave correctly under a certain circumstances (different lighting, view angle & distance) I don’t mind faking it entirely.

For example, with the subsurface gradient Fresnel Ramp, it would be way more accurate if It behaved based on light direction like a real subsurface scattering would do, but since this effect is fairly subtle, I can get away with a simple fresnel gradient. This might not work for character with strong lighting cast upon it. The SS color has to be falloff based on lighting direction, not view angle. But for this case this is just good enough.

Whenever I decided to add a layer of detail to my shader, I would need to test if it would look correct and how it behaves under different lighting conditions and view angles. Sometimes an effect looks really nice if it’s static image and viewed from the front, but it would look super weird if viewed from side like the fuzz effect for example, so I have to balance between what looks best vs what works best or even need to figure out a new way to make that effect from scratch again with different approach.

Structure

Some approach/setup might not be accurate yet, but I have all the essential elements drafted out to have a nice result so I can always come back to refine this part later. For example, in the future if I can come up with a better way to do the custom reflection map or subsurface gradient I can update these Material functions which are already sorted and stored in my own library – which is ready to be reused and upgraded.

Balance

It’s important that once I finish adding all the fancy details into the shaders I step back and judge if I am going too far from the original goal, for this particular project I want to keep it as close to stylized digital 2D painting look as possible so I need to adjust anything that could make the whole thing look unnatural or too far off from the style.

For example cranking up the normal / bumpy of the skin surface might make it look too much of a digital 3D piece.

Facial Features

In my old projects, I just threw in my hand-painted texture assets into UE4, added some lights then that’s it. The whole thing just doesn’t feel right, hand-painted texture assets might look good when viewed unlit and have their very own style from just the unlit color texture, but when lit, there will be big troubles as the color texture is not the only one that affects the model anymore. There are shadows, highlights and reflection/specular in the scene, you will lose the details painted onto the texture and the color texture will be revealed as being literally painted on.

This D.va is one of my old projects when I haven’t digged much into UE4, so everything is very basic setup. So in this project – Exparia, I take extra care to make sure that the lighting/shadow/ reflection reacts correctly on the model. Especially in the face areas, I pay attention to face’s elements like eyes, nose and mouth. I can spend the whole day on adjusting texture, editing model, tweaking shader back & forth, little by little just to get it right.

In my workflow from beginning towards the end, everything is pretty much straightforward except for the refine part. The time it took me to refine might be as much as all the previous steps combined but it was worth the effort. There is no straightforward solution of how to get it right from the first time, some time I even had to go back to model a new part, change the model structure of an entire part to make it work together with the shader I just created.

Eye

At first, I have it as a merged eye (no separate eyeballs) but later I had to break it down to several different part like below, so I had more control over individual elements of the eye like reflection, hi-light, rotating eyeball etc.

Below is the graph of the eye shaders:

Eyelashes

In the beginning, Eyelashes are painted directly onto face texture, but later I had to extract it into a separate sheet of mesh, and for the final result, I had to duplicate the eyelashes meshes to give it more volume. I tried using translucency shader as much as possible because its AA alpha mask is far better than Masked shader type. The only setback is its sorting priority, but since the eye has a really neat concave shape, I can get away with sorting in this case. Other alternative option is using Dither Opacity Mask shader which also gives you a nice AA result but it’s not good for thin streaks or semi transparent and moving surface.

Lips

There are no micro details for the lip area at first, as you can see I have sculpted much detail on the lips in Zbrush as I don’t really want to go that far into detail while sculpting. I feel much more comfortable with adding details later in Photoshop, than sculpts it straight solid in my sculpt because I have the freedom and speed to try different types of details to see if it works, same goes for the skin spore & details – I prefer adding these details in texture rather than sculpting.

On a side note, sometimes I need to animate one of these elements as a single piece and render it in super closeup to see if all the details can survive the changes in animation/deformation & lighting/view angle like lips animation, eye close & open, etc. Animating or making things deform in UE4 is pretty simple, I just prepare a series of blend shapes of the meshes. Why blendshapes? Because it’s much faster to test and much easier to control the deformation,, than joint skinning animation. Then animate the Blendshape in Maya, export the whole sequence into UE4 then do a quick test sequence render. 

Rendering

For this project “Exparia”, I don’t hold back in terms of setting because the whole scene will be used to create a small cinematic scene later which can be rendered to movie using UE4 level sequencer or scaled down in terms of quality if I want to run it as a real-time demo at a decent frame rate. I max out every possible render setting in UE4 because I don’t want anything to hold me back when developing a visual piece. Optimization will come later. Below are some tips & tricks to increase the UE4 render settings I used:

I disable pre-computed lighting so the entire time I will work in realtime dynamic lighting, there’s no waiting time for the lighting to be built.

As I am only working on the refining & ldev atm, I tend to keep the lighting as simple as possible, I used the BP_lightStudio blueprint asset in starter pack as a base scene so I have one directional light, a skylight and a HDRI dome to start with, so no fancy spot light, or area point lights, etc to avoid any confusions & misleading when troubleshooting shadow & reflection later on.

Dzung Phung Dinh, Founder – Thunder Cloud Studio

Interview conducted by Kirill Tokarev