Creating an Old House in 3ds Max, ZBrush & Unreal Engine 5

Introduction

Hi there! My name is Anna Stopina and I live in Toronto. Right now I am in my final year of the Game – Art Program at George Brown College. Game Environments always amazed me, so when I started my time in college, I already knew I wanted to be an environment artist and I never doubted my decision.
 
After my second year in college, I wanted to work on my own scene over the summer. Unfortunately, I didn’t have much progress with it because I was constantly making adjustments and tweaks, solving problems that were not the priority at that stage of the project.

I realized that I needed help with my work approach and started looking for professional help. On social media, I stumbled upon the SkillTree platform. Since I am a fan of Billy Matjiunis’ artistic vision and I was also familiar with his effective teaching approach from the college, I decided to get mentorship from him.

My main goals for the mentorship were to improve my skill in composition, work on the color palette of the scene, along with visual storytelling. I also wanted to establish visually interesting and appealing cinematic shots. 

From a technical standpoint, I wanted to get better in texturing (both in the quality of the textures and in selecting the method of texturing).

Planning and Blocking

From the beginning, I knew that I wanted to create a scene with a strong nostalgic feeling and with an emphasis on storytelling. 

I took a lot of inspiration from this collection of photographs.

I started with a rough blockout of the main shapes to see how the space feels. I also placed a couple of cameras to see the space through their lenses.

Initially, I started with creating what, at that moment, I considered the most important props – a parlor stove and a chair. I also took a little bit of time and sculpted wood for the fireplace. 

With Billy’s guidance, the approach has been shifted, and the first thing I did was to organize my reference board (props, materials, lighting, mood, particles, etc.), block-out everything, including small props.

When I did it, everything became way clearer – what to focus on first, where to put more details, what materials need to be created, what props should be modeled, etc. Here’s a small portion of my reference board.

As a placeholder, on the wallpaper, I used a pattern extracted from a reference image, I don’t remember its source, I had it on my PC for a long time before I had a chance to use it.

Modeling and Texturing

The next step was modeling and texturing the props.

All scene props were divided into two parts in terms of texturing, which affected their further creation: props that would use unique textures and tileable textures. More complex props would use unique textures, more generic would use tileable textures.

In my project, I had to break the rule in some cases because I don’t have a sufficient material library yet. Therefore, it was significantly faster to give a mesh an individual texture instead. Often, several small props shared one texture set. 

The main advantage of texturing a prop with a unique texture is endless possibilities to give a prop a more elaborate story, more character by adding several layers of unique details, you’re not limited by the number of textures you can use in the engine. The main drawback, however, is that it is a time-consuming process. 

On the other hand, the main advantage of using tileable textures and masks is its speed. A lot of the time was spent in Substance 3D Designer creating decent textures. As soon as I had the texture I needed on hand, I was pretty much halfway there. 

For objects with individual textures, I used a standard high to low poly workflow. I would model a base mesh in 3ds Max, sculpt details in ZBrush, make a low poly version in 3ds Max, and finally bake maps and texture a model in Substance 3D Painter. Here, I should mention that since it was my personal project, I used some extra polygons for my low poly to achieve a better silhouette of a prop.

I only scratched the surface of RGBA Masks workflow before and I wanted to learn it more in-depth and practice it in my project.

To create tileable textures, I used Substance 3D Designer. The chosen texel density for the scene was 2048x2048 for 1.5 square meters, and all my materials were created with this texel density in mind.

Since it’s a similar approach for all props I textured with tileable textures, I will take the dresser from the scene as an example.

I divided my texturing process with tileable materials and RGBA masks into three stages.

1. Create a base mesh. Create materials. Create a proxy mesh for Substance 3D Painter.

Before making anything, I had to decide what would be modeled and what would be added by using textures and decals.

For the dresser, I decided that the body will be a mid poly model, it will need clean and damaged wood textures and some damaged wood decals.

However, since I wanted to give some extra details for the dresser pulls and decals, which I would not be able to achieve using this method, I assigned them separate material IDs. I had 3 material IDs for the dresser in total.

The body of a dresser I modeled in 3ds Max. To give my base mesh a high poly look, I chamfered all my hard edges and applied the Weighted Normals Modifier.

For the pulls, I did a sculpt in ZBrush and baked the Normal map in Substance 3D Painter. 

After the dresser was modeled and I was sure I would not need to make any changes in the model, I started preparing the mesh for blend masks. I duplicated my UV to the second UV channel and packed it there in 0-1 UV space. I duplicated my mesh, moved my UV from the second UV channel to the first, and exported the latter for further baking in Substance 3D Painter.

2. Apply blend-masks.

I did a bake in Substance 3D Painter using my base proxy mesh. Using information from the black and white channels as masks (Metallic, Roughness, Ambient Occlusion, Opacity), I determined where I want to have extra details – in this case, damaged wood and dirt. Then I used channel-packed masks to expose this information in the engine.

3. Apply details that cannot be added with tileable materials.

For decals, I composed an atlas in Photoshop that I used for several other models. Since I wanted to achieve the most realistic look, I used reference pictures of chipped veneer furniture to compose my atlas of damaged veneer. It is worth mentioning that for other types of decals (they were used on the bench, the entrance, and the chair), I sculpted damaged wood in ZBrush and then baked, textured, and masked in Substance 3D Painter. 

To add decals to more complex shapes, like the dresser’s leg, I duplicated the geometry where I need a decal to be, very slightly offset it to avoid Z-fighting, and mapped it on my damaged wood decal atlas. On a flat surface, decals were just small mapped planes sitting on top.

This is my final stage of a prop:

For the modeling itself, I didn’t do anything special. The only thing I always make sure I do is chamfering hard edges and applying weighted normals modifier in 3ds Max. This works like magic. Those several extra polygons make a model look significantly better. 

I also try to bring my model to the engine as early as possible and update it several times as I progress with it. I found that the way my model looks on its own in 3ds Max could be quite different from how it looks in the engine and as a part of the scene. These several minutes for the update could potentially save many hours of fixing stuff and a headache. 

Also, an important thing to keep in mind is the direction of your texture while unwrapping models. To make my life a little bit easier, I decided that all my wood materials would be oriented horizontally.

Working in Unreal Engine 5, Lighting, and Post-Processing

The biggest feature of UE5, Lumen, worked perfectly for my scene. Initially, I started the project in UE4 and just out of curiosity tested my lighting in UE5, and it was marvelous. It instantly gave my scene the feeling I was after.

However, when I was lighting my scene, I encountered a couple of issues.

First, the light didn’t pass my curtains through and the shadows were off on some objects. The solution was extremely simple – I made sure I didn’t use virtual shadow maps in the Project Settings. 

Second, Lumen did not work properly with the HDRI dome. To fix this, I disabled "Affect Distance Field Lighting" in the Editor Sky Sphere. I found out about this from William Faucher's YouTube Channel.

In terms of lighting, I mostly relied on a directional light. I also added a couple of very subtle highlights. For example, I added just a subtle orange lighting spot in front of the stove to give it an extra touch of warm orange light. I also added a little bit of light to the corner area because originally it was slightly dark.

I didn’t do anything special for post-processing in the engine, just simple color grading. However, to enhance the cinematic feeling of the scene, I did the final pass in Photoshop. My favorite thing there was adding a noise overlay layer (setting a blend mode to screen and opacity about 30-40%). It adds cinematic graininess and ties everything together.

The new lighting system of UE5 significantly streamlined my workflow. My lighting system was fully dynamic, and I didn’t have to spend time on baking lighting. Bouncing light gave my scene a more natural look. It felt great to make changes on the fly. 

Final Thoughts

Getting mentorship was one of my best decisions in 2021. It was beneficial in every aspect I can think of. Feedback from my mentor was extremely helpful. Billy pointed out things I hadn't noticed before, showed me new approaches, and really boosted my morale.

Working on this scene helped me detect flaws in the way I approach my environments. I know where I should be extra careful and where I should stop overthinking and overcomplicating things (sometimes you have to just stop, take a break, or even call something done).

In conclusion, I would say that planning and polishing are the two most important stages. I learned to plan everything carefully: plan your composition, plan your materials so you can reuse them, plan your atlas so you can map your details later, etc. It might seem like you’re not working while planning, whereas, in fact, you are doing one of the most crucial things. 

Also, I learned that it’s important to get to your polishing stage as fast as possible. Adding details is a hard and long process, but this is what makes your scene interesting and full of life. 

As I mentioned at the beginning of the article, this is my final year of college, and I can’t wait to start my career in games. 

Please feel free to contact me through email or Artstation if you have any questions or comments. 

Sincere thanks to my classmate Amber Pennington for proofreading the text.

Anna Stopina, 3D Environment Artist

Interview conducted by Kirill Tokarev