Making an Animated Stylized House Diorama in Substance & Unreal

Introduction

My name is Taylor Harris and I am a 3D Environment Artist from Seattle. In the mid-2000’s I earned a Bachelor of Arts in Animation but decided my interest was in making environments rather than character work. For 14 years I worked for a company that makes 3D virtual tours of sports stadiums and concert venues. Our product allowed a sports fan to tour their favorite team’s stadium virtually before buying tickets to a game. However, as a life-long gamer, I wanted to pivot my career towards the world of game art. The impact of COVID-19 on the athletics industry accelerated this plan.

When you work at one company for as long as I did, it can be a handicap to settle into the practices and workflows that are unique to that company, and lose sight of how the industry as a whole operates. To combat this, I have been taking workshops through Vertex School to brush up on my skills and make sure I am aware of the current trends and techniques used in the game industry. While I worked on this project, my primary mentor at Vertex School was Lucas Annunziata, a Senior Environment Artist for Overwatch 2 at Blizzard Entertainment. 

Working on the Neon Eye Project

For this project, I started my search for a new concept piece on Pinterest with a very vague search term like “environment concept art”. I like Pinterest because it is easy to fall into a rabbit hole that leads you to some interesting artwork. If I click on a cyberpunk illustration, the recommended images on the following page will all have cyberpunk influences, but if I click an image with gothic architecture then that choice will be reflected in the resulting recommendations.  

So, I followed a path of more fantasy-themed pieces, saving the ones that stood out to me, and then eventually landed on the piece called Neon Eye by Natália Chromá. I knew pretty quickly that this was my favorite of the pieces I had collected. I loved the color scheme, the architectural forms, and I felt like it had a personality to it, as if I could already envision the characters that would visit this building and their stories. Lately, the environments that inspire me the most tend to be happier settings that feel welcoming to the viewer, and I felt this piece matched my tastes. 

I reached out to Natália on every social network I could find her on and used Google Translate to write my request to use her concept in multiple languages, as I did not know whether she spoke English or not. I was very happy to receive her approval via Facebook.

Modeling and Texturing

Something that is very important to me when I make a piece like this is staying true to the original concept. I am specifically trying to show potential employers that I have an eye for detail, and if they have a concept artist on the payroll I want them to know that I will honor that artist’s hard work by keeping it true to their vision. To that end, I have a technique I use to assist me in the initial block-out phase. I use tools in 3ds Max to match the perspective and focal length of my camera to the concept. Doing this takes a lot of the guesswork out of matching the proportions in the concept and speeds up the modeling process.

I have a 10-minute video on the subject for anyone that would like to see me walk through that process.

Doing this takes a lot of the guesswork out of matching the proportions in the concept, and speeds up the modeling process.

There were two elements to this piece that proved to be the most challenging: lighting and texture detail. In the original concept, the lighting for this scene was very soft and overcast. However, because I am trying to make this look like a piece of game art, I felt that the type of game that would use this kind of aesthetic probably would not have demanding rendering requirements or advanced lighting systems. For that reason, I simplified the lighting with a more direct light source. I tried to compensate for this lighting difference by having some of the material work and the bloom settings make up for some of that softness.

As for the texture detail, I felt that the concept successfully implies different types of materials, but a 3D replication would need more detail than is present in the concept. I would estimate about 70% of the time I spent on this project was addressing this challenge, constantly pushing and pulling the materials in different directions trying to get a look I liked, while making them all play well together. Though, fortunately for me, Natália’s concept already had a color scheme that I loved, so that made my texturing work a lot easier. The majority of the material work was done in Substance Designer. I used it to create a brushstroke pattern and incorporate those brushstrokes into different materials. This helped give different materials a commonality that made them look like they all belong to the same setting.

Some of the more detailed forms like the dragon heads were done in Substance Painter, the foliage was hand-painted in Photoshop, and the neon lights were simply done in Unreal Engine. As far as the unwrapping of the diorama for texturing goes, I cannot say that I have any clever tricks or pioneered anything new. I did my best to make use of trim sheets on as many materials as possible and used 3ds Max to unwrap the model in accordance with those trims.

By the way, this was actually my first time using Substance Designer, and there was a bit of a learning curve to overcome. The first materials I made were the orange brick pattern and the green tile roof. I probably made each of those materials five or six times, experimenting with different approaches as I learned Designer. As a new user, it seems as though many of the learning resources for Substance Designer are aimed at realistic art styles, but eventually, I found resources that helped me visualize how to use procedural methods to create materials that had a painterly feel. Jimmy Malachier was one of my strongest influences in this regard.

Working on Animation

Throughout the production of this piece I knew I wanted to add animated elements, but I was not really sure what to incorporate. I started with the more obvious elements that could be animated, such as the foliage blowing in a breeze and a light function to portray clouds moving in front of the sun. After that, I had to get more creative with finding elements that could bring some motion to the scene. 

After the wind effects, the next animated elements I added were the moths that fly around the neon lights. I used Niagara VFX System in Unreal Engine to accomplish this, and to be honest, it was my first time using Niagara, so I am pretty sure my methods were not optimal. I made a simple animation of a moth flapping its wings a single time in 3ds Max and imported that as a skeletal mesh into UE4.

To govern their spontaneous movements, I used Attraction Force, and I set their attraction strength to a curve that oscillates between positive and negative values at an irregular rate. This would cause the moths to be repeatedly repelled and attracted. Then I combined that with a mish-mash of other forces to add some chaos to their flight paths, such as Vortex Force and Curl Noise Force.

Next up was the smoke coming out of the chimney. The method I used for this I learned from a video on Stylized Station by Johannes Burström. This part was fairly straightforward: Model a relatively low poly smoke object, enable tessellation and use some animated smoke maps to displace that mesh. I lowered the influence of the mesh displacement near the chimney, so the mesh wouldn’t clip through its sides as it deformed.

The opaque stylization of the smoke was actually a happy accident. I tried doing different types of translucency effects but found they didn’t play well with the white border on the edges of this piece (which is captured in-engine). I changed the smoke to an opaque material and not only did it fix that problem, I actually thought it fit the aesthetic of the piece better.

I had been intrigued with the dragon fixtures on the front porch because it is not obvious what their purpose is supposed to be. Are they the downspouts of a gutter? Mailboxes? When I looked at them, I was reminded of an effect from an animated series I enjoy called Tales from Alethrion. There is a momentary shot of an explosion of colored smoke shooting out of a cave, and when I looked at these dragon fixtures I imagined a similar effect.

Animation by @mikkelmainz

I imagined this building could be the shop for a friendly wizard, and perhaps when they mix potions inside, there would be spontaneous bursts of colored light and smoke which could shoot out of these dragon fixtures. This created a lot more opportunity to breathe life into the scene with animated effects. I also could not decide which color of smoke I thought looked best, so this gave me an excuse to have the explosions cycle through different colors.

Attempt 1: I wanted to use the same method that I used for the smoke coming out of the chimney, but this turned out to be much more difficult. This smoke needed to appear as though a sudden burst of energy expelled it from those pipes, and using an opacity mask to gradually climb a static mesh object was not selling the dynamic force this effect was supposed to have. The opacity mask also did not have desirable results as it climbed the geometry to reveal the shape of the smoke.

Attempt 2:  I had success using Niagara on the moths, so I decided to try and make this smoke effect in Niagara as well. The results were interesting, but still not what I needed. I really liked the physics of how the particles moved, but the visual aesthetic didn’t match the smoke coming from the chimney. I tried altering the particle size and particle count, but everything I tried resulted in a very soft and fuzzy-looking smoke effect.

Attempt 3: I felt that the solution to this problem would require me to have an actual solid mesh representing the smoke, rather than particles, I just was not sure how to accomplish this. I experimented with using BlobMesh, a metaball-like simulation that combines amorphous mesh shapes together when they get near each other (think lava lamp blobs). The problem with this is that, as far as I can tell, I cannot export this type of effect to Unreal Engine, nor UV map a BlobMesh, as the vertex count of the object is changing every frame.

Attempt 4: I was almost out of ideas when I stumbled upon a solution. I thought back to my days in school for animation and was reminded of morph targets, or blend shapes, to control facial animations. I could set up morph targets for smoke plume shapes in different phases of its animation, almost like traditional 2D animation keyframes. I would be able to unwrap this object and preserve its mapping as it deformed.

It looks pretty goofy and artificial when viewed in isolation like this, but I was confident that with the right material I could make this work. So now that I had the geometry issue solved, it was time to jump into Substance Designer and create an opacity mask that I could animate. I used the same technique as the chimney smoke, but with an additional spiral-patterned opacity map. I am really happy with the results because it looks like there is turbulence and air currents within the smoke as it dissipates. The spiral pattern was created in Substance Designer, and composited into the smoke material in Unreal Engine.

I used Blueprint to time a flare of light intensity on a 10-second loop with each explosion, but it was still missing something to really sell the intensity of these explosions. I decided there needed to be some dust shaken off the sides of the building with each burst. For this, I would use a flipbook animation to animate a cloud of dust falling along the edges of the building.

As a challenge to myself, I wanted to see if I could make a Substance Designer graph that could generate the different frames of a flipbook for me, automatically. The basic principle was the following: I would define what the first frame of the flipbook should be, then adjust the parameters of that SD graph to define what the last frame should look like.

I could then define how many rows and columns I wanted my sprite sheet to contain, and it would basically ‘tween’ as many images as it needed to fill between that first and last frame. For example, if I tell the graph I need a 3x3 sprite sheet, it would know that I need 9 total frames (first and last, plus 7 in between). If I tell it I need a 5x5 sprite sheet, it would generate 25 total frames (first, last, and 23 in between). 

Once I had this flipbook material made, I then took a cloud material from one of my other materials and used it as a macrotexture to mask out portions of this dust effect, as it was looking very repetitive otherwise.

The main lesson I learned when working on these effects had to do with syncing them all together. In total, I had to sync all of the following elements: three different light sources; a dynamic emissive value within a material; an animated skeletal mesh; changing smoke colors within two different materials; an animated opacity map on the dragon breath; a flipbook material for the dust; a displacement map surge in the chimney smoke with each explosion. All of these elements needed to sync up, which can be challenging when you control some of these things within a material, others within a Blueprint, and so on.

My advice to anyone who finds themselves in a similar situation would be to have a physical pen and paper handy to keep track of the timing of all your elements. In my case, the complete animation loops every 10 seconds, but certain elements such as the smoke from the dragons' heads only last 3 seconds. I learned that it was easier to make the dragon breath animation also a 10-second loop, even if 7 seconds did not contain any motion. That way I would not have to worry about starting and stopping a 3-second loop repeatedly while making sure it still was in sync with the 10-second sequence.

Lighting and Rendering

The lighting in this project was a very iterative process. I roughed it out at the start, and made certain design choices like “How strong do I want the directional light compared to the ambient light?”, but I was constantly adjusting the lights as I progressed. Oftentimes if something doesn’t look right with your materials, it’s actually a lighting problem or vice versa.

I've learned to rely heavily on using the ‘unlit’ shading mode in Unreal Engine to assess whether a contrast problem is the result of my materials or lighting. If the unlit mode looks like it has high contrast, then your albedo maps are probably too dark and/or too light and you’ll need to push those values closer to mid-tones. Because, frankly, the unlit mode *should* look kind of muddy and ugly.

When looking at the neon lights in the concept art, I noticed that even though the light reads as purple, the surface is actually white with a dark purple outline. I tried to replicate this effect by using a Fresnel material to separate the ‘core’ of the light from the outer edges, and I could assign different color values to each. By the end of the project I decided that I would be introducing a fair amount of bloom, and that sadly overwhelmed some of the subtlety I tried to put in the neon lights, but I did make use of the same technique on other elements such as the glass bottle on the ground, where I also wanted to simulate an outline around the silhouette of the object.

The Tricky Part

The biggest challenge of this piece was making sure that each element looked like a part of a cohesive whole. There were several materials I completely started over on four or five times just because they did not quite fit with other materials, even if I liked how they looked in isolation.

In every piece I do I try to challenge myself to incorporate something outside of my comfort zone. In this piece, I was afraid to create the foliage for a long time. I received some very good advice from Anya Elvidge that completely changed my thought process on how to approach foliage. Anya had explained the process of creating foliage as being very similar to how you model something more architectural: for a building, you would start by greyboxing out the primary forms, and once you have those proportionally correct, you start to add the secondary forms, and finally the smaller tertiary details. Foliage can be approached similarly.

I had been struggling because I had been overthinking all the elements of a plant and incorporating them all at once (painting the stem, connecting leaves to the stem, painting a spine on each leaf, shading every leaf). Instead, she recommended I start very simply by hastily painting opacity maps with some vaguely leaf-shaped blobs in the form of my plant. I did not need to even worry about any kind of stem or structure to the plant, I was only concerned with filling out the volume of space that this plant was going to take up. From there I could map these Opacity Maps onto some simple geometry and place it in Unreal Engine to see if it is looking right.

Once that is looking right, I can go back into Photoshop and start painting in secondary details such as the spines, or refine any of the leaf shapes that might have been too sloppy the first time around.

Looking at the concept, I felt the plants could be broken up into four different color groups: yellow, purple/grey, and a couple of shades of green. I felt like it would be a waste of time to make four unique color maps for these plants, so instead, I made a utility map that would composite different color values together inside of Unreal and hopefully allow me an infinite amount of color possibilities using just one map. So I broke down the shapes I could see in the concept art, and using the RGB channels in Photoshop I assigned each channel to a different form that I could see within the concept.

In this case, the red channel would represent the alpha map for the entire plant and I would use this to mask in the primary base color. In the center mass of these plants, you can see the artist used slight color variation to imply the complexity of overlapping leaves, even when those leaves are not clearly defined. In the green channel, I painted some overlapping brush strokes at low opacity to try to reproduce this implied complexity. I would use this channel to mix in a variation of the base color. Lastly, I used the blue channel to paint a bold edge detail around the silhouette of the leaves. I tried to make this outline irregular. This channel would mask a darker edge color around the plant. Using these three channels I could swap in different colors to make as many color variations as I needed.

The Power of Reflection

In conclusion, I am pretty happy with how this piece turned out. Like any project, there are many aspects of it that I would try to improve on if I was to do it all over again, but sometimes you have to take those lessons with you and apply them to your next project rather than re-hashing one piece until the end of time.   

If I can leave you with some parting advice, I would like to share something that helped me greatly while working on this project, and that is how I managed my mental energy and state of mind. It is easy to get burned out or get frustrated when working on a problematic area and just abandon a project.

Periodically I would stand up and look at the piece from afar, and I mean literally walk to the end of a hallway and look at it from different distances. I would then ask myself two questions about the piece. Firstly, in its current state, what part of the piece is the weakest, what element is holding it back the most? Secondly, of all the work I have remaining to do, what aspect would be the easiest to add and make the biggest improvement?

Depending on my state of mind at the time, I would pursue one of those questions. If I am feeling motivated and positive, I will take a stab at tackling the most difficult part of the project or something problematic that I may have rehashed several times but not yet found the best solution for. If I am feeling close to burnout or frustrated with the project, then I will try to find the simplest element to work on that yields the best bang for my buck. It is surprising how sometimes a relatively simple task can make a world of difference in how you view your progress, and once you complete it, it can leave you feeling refreshed and excited about the project. My mentor on this project, Lucas Annunziata, called this looking for ‘easy wins’.

Thanks for reading through this whole process with me, I hope you found it helpful or interesting. Feel free to reach out to me on ArtStation if there is any of this I can elaborate on further.

Taylor Harris, 3D Environment Artist

Interview conducted by Arti Sergeev