LogInFelix Voigt shared the details of his atmospheric project At the Sea inspired by Simon Stålenhag art and based on his book The Electric State.
Felix Voigt shared the details of his atmospheric project At the Sea inspired by Simon Stålenhag art and based on his book The Electric State.
Introduction
Hi! My name is Felix Voigt and I have just graduated from the University of Applied Sciences Darmstadt, Germany. I studied Animation & Game, learned a lot about many aspects of animation movies and game development and very quickly fell in love with the creation of environments for games. During this time, I also had the opportunity to work as an intern at King Art Games and as a freelancer at metricminds where we created many in-game cutscenes for Darksiders III.
My journey began around 5 years ago when I was still in school and didn’t have a clue what to do afterward. With pure luck, I discovered that there was a possibility to attend to game-related studies in Germany and I began to dive deeper into that area. I decided to create a game myself (that idea didn’t seem as crazy to me back then) and downloaded CryEngine. I was pretty surprised when I realized that all the cool Crysis assets were not included and that I had to make them myself. From that point on, I started shaping my inner 3D artist.
The Idea of the Scene
I had to decide what I was going to do for my bachelor thesis and I knew that I wanted to do an environment that’d allow me to improve the skills and get a nice piece for my portfolio. From my past projects, I knew that designing a whole environment as well as producing it is very hard and time-consuming. I wanted to focus on improving my modeling, texturing, and artistic skills.
At that point, I had followed Simon Stålenhag for quite some time so choosing his art as the basis for the project was an easy decision. By relying on his illustrations, I could both focus on producing good assets and improve artistically. I wanted to understand why he did what he did, how he achieved this mood in his paintings, how he worked with the shapes and composition. For me, it was sort of a master study, I just worked with another medium.
Choosing the Concept
I bought The Electric State book after I made the decision to base my thesis on Simon’s work and became even more impressed. The whole book is full of mystery and darkness. As a reader, I constantly tried to understand what had happened, what pushed the world to the edge, why the characters were making the road trip and so on. I decided to convey this experience through my art and stay true to the original.
When finally holding the book in my hands, I also had to decide which illustration I wanted to base my work on. I considered around five pieces based on multiple criteria.
- Firstly, and quite obviously, I chose the ones I liked the most.
- Secondly, I chose right from the beginning to create an animation out of the environment, as it is a great way to bring it to life even more. It had to be a picture that allowed some motion. This was also not difficult as many pictures had cars, streets, and vegetation that could be moved by the wind.
- Thirdly, the scene had to be released in 4-5 months during which I also had to write around 80 pages of research paper and bachelor thesis. That meant nothing with too many crazy models and textures so that I could focus on producing less with higher quality.
- Fourthly and lastly: no characters. My characters always become quite derpy, plus diving deeper into human anatomy, character animation, and texturing for this project was way out of scope. I didn’t want to endanger my scene through any abominations crawling out of the uncanny valley.
The scene I chose met all the criteria and also had four illustrations of the same place which meant plenty of source material to work with. They were the culmination of the story and not accompanied and explained by the text. I was really caught by that scene and could not resist to take it.
Fog
The fog is clearly a core element of the scene. It is essential for the mentioned mystery as it hides the distant shapes such as the rocks in the background and underlines their silhouettes. I worked on the lighting and the fog from the very beginning, right after the first basic block out. Both were important for determining the look of the scene and for the materials. I frequently tweaked the fog settings as there was a fine line between too thick and too revealing.
I also used a simple setup of volumetric materials to add some more volume fog where I needed it. They were used specifically to hide the shape of the big server tower in the background and to create a gradient between the sky and the water on top of the horizon line. The useful thing about the material was that I made the fog dependent on the bounding box which gave me the ability to scale the fog how I needed it.
The ocean came initially from Houdini where I simulated the water’s behavior on the beach and also received particles from the whitewater simulation. I did a small experiment and was able to utilize Houdini’s Volume Texture Tool to render out a volume texture of the foam and spray. I did this 64 frames and was able to receive a flipbook texture containing flipbooks of the volume slices. I used that in conjunction with Ryan Bucks’ GDC Shaderbits Plugin to create an animated Volumetric Fog in Unreal for the foam and spray. The result is very subtle and I am not sure if it adds anything noticeable to the scene. Despite that, I had to use an 8k map to get a fog of a decent resolution. Nevertheless, I wanted to mention it here as the technique itself worked and could be more useful to someone else.
Modeling of Skip & Car
The most difficult models were the robot Skip and the car. Skip was easier as there were many illustrations of him, some from a very close distance which meant that I had plenty of source material. Sometimes, the illustrations contradicted each other as they displayed the robot differently, like how the single metal shells are attached or welded together. Here I had to simply decide which one I preferred or which made the most sense as many differences were minor and only noticeable in direct comparison. It was also fun to figure out how he should move and how to design some of his joints to enable flexibility.
Car was tougher as cars are incredibly complex objects that have to be functional. I wanted to include the interior which had to have the right scale and shape for humans to interact with and fit into the outer shape of the car. Luckily, Simon Stålenhag decided to use a real car for his pictures and I was able to trace the model down. To achieve the intended result, I gathered solid references which is probably one of the most important steps in modeling. The actual process of modeling was pretty straightforward as I followed a basic low poly/high poly workflow in 3ds Max. I worked a lot with the object references rather than their copies. Therefore, I was able to create base models with a nice topology and sensible resolution which I built the high poly and low poly models on. Working with referenced models allowed me to have a higher degree of flexibility in the scene as I could go down in the modifier stack and make changes that affected both high and low poly models without repeating steps or losing progress.
Ocean
Before this project, I had never used Houdini and needed some time to get into the program, but it was clearly worth it. When choosing the scene, I thought that the ocean would play a crucial part: if it wasn’t able to convince the viewer, the whole scene would suffer from it. Therefore, the first thing that I did was to find a solution for convincing, loopable beach waves and to test it. For me, this was the scene element with the highest risk. If I could prove early in production that the technique was achievable, I could work more relaxed on the project. Additionally, I was still able to switch to another scene if I realized that the ocean was not feasible.
Luckily, I was able to utilize the workflow used by Matt Vitalone for Call of Duty: WWII. The basic idea was to make a beach tank simulation in Houdini. A plane is projected onto the resulting mesh and compared with its unprojected version. The difference results in a displacement map. Additionally, I generated a normal map from the simulated waves and a foam mask from the simulated foam. In the final step, the textures were made tileable and loopable, and put into one flipbook texture for each channel (displacement, normal, and foam). This was not only done for the simulated beach tank, but also for a plane of tiling open ocean. One problem was that the generated textures became quite big: as one texture covered around 64m*64m in Unreal to avoid obvious repetition, I needed a 1k map for that area. Therefore, I rendered 64 frames into the texture to get a resolution of 8k for the flipbooks from which I needed three (one for each channel). These textures took up a lot of space, but I decided that the ocean was important enough to take up such a huge space in the memory. Another problem was the number of frames: one loop had to take around 5 seconds to look good. With the texture only having 64 sub-textures, the animation would result in a speed of around 12 frames per second. That meant that the performance of the texture stuttered and it didn’t move fluently.
The three animated textures: displacement, normal and foam mask.
As I didn’t want to make a change in Unreal to enable the engine to use 16k maps (despite my gut telling me that it was not a good idea to use such big maps), I decided to build my own version of Unreal’s flipbook node. The special thing about it was that it not only jumped from picture to picture like its original version but had two texture samplers: one for the current and one for the next step of the flipbook. With increasing time, it would lerp between these two until the first one was fully blended in. Then everything stepped one picture forward and the process started again. With this method, the flipbook could blend between single pictures instead of jumping in-between and create a smoother animation result.
When I knew that the ocean would work, I iterated mostly on the quality of the simulation. That included the simulation’s resolution which was very bad in the beginning. I also tried to improve some settings to speed up render time. The problem was that I had to cover a large area of 64*64m with a high texel density (6cm per pixel from a 1k map!). The number of necessary particles in the simulation was enormous and my PC at home, as well as in my university, just couldn’t handle the simulation. In the end, I decided to work with the render farm from GridMarkets which delivered pleasing results in a short amount of time while not being very expensive. It took some time to implement their nodes into my node structure correctly but the results were definitely worth it.
Sand Material
All of my material work was done with Substance products. In the beginning, I made a very fast iteration of the sand that allowed me to throw something into the engine. At that point, I wanted to fill the scene with some basic materials to make the first landscape painting and to get some colors and roughness values into the scene. This was very helpful later during the light setup.
Doing the lighting so early in the process was something new to me and led me, in terms of the materials, into the wrong direction first. After building the first quick-and-dirty version of the sand material, I kept relying on this version and iterated further on it without noticing that I got dragged into the details way too much. My first version was basically nothing more than some high-frequency noises with color. One of my friends was able to point out the problem and I realized that my texture consisted mostly of a detail layer, but did not feature any large or medium scale patterns. The fault also laid in my reference collection, as it was mostly compromised of detailed sand structures rather than larger structures and forms. Additionally, sand contains more elements than simply pure sand, especially on a beach where pebbles in different sizes or sea life remains can be found.
I had two versions of the sand: dry sand and wet sand. The dry sand had no proper directionality, and in the scenes, I combined it with a separate material of gravel. I did this to relieve the landscape of repetition without creating huge textures. The wet sand, on the other hand, had directionality: I decided to add more patterns to it and to combine it with some channels that funnel the water that flows back with the waves. I also added some shells built in a generator to both sand versions. In the wet sand, I implemented them in such a way that the sand built a heap where it would flow against the shell and a trench where it gathered under the shell. With this subtle effect, I could ground them better in the material. In terms of the roughness, I decided to make it glossier than what would probably be considered realistic, so that it would reflect its surroundings much more. To push this effect even further, I decided to insert one of Unreal’s costly planar reflection captures, which gave the wet sand a look similar to the original artworks. Additionally, I also layered a foam material on top of the wet sand to portray the foam which the waves leave behind.
On this project, I wanted to really get as much feedback as possible from other people. I knew that this scene would take a few months and that I would work alone on it. By getting other people’s opinion, I was able to see things that I simply wasn’t able to see anymore. Sometimes it was something obvious, like with the sand texture, but when you are standing too close to a subject for a long time, you stop seeing the bigger picture. Also, it is sometimes a good thing to be bold and just throw away what you are working on if it does not work. I know that I am sometimes anxious to waste time or throw things away after I’ve put some time into them. Starting over is hard, but enables you to tackle a problem from another direction or avoid mistakes you have done earlier in the process. Another thing I learned to do in Substance throughout this project was to build my own library of custom nodes and encapsulate things like the shell generator into their own graph. These measurements simply speeded up the creation process and helped organize the graph, as it became easier to read and maintain.
Lighting & Colors
To create the scene’s look, I used Unreal’s Volumetric Fog in combination with a stronger skylight and a very soft directional light coming straight from above for soft shadows and specular reflections. The soft light helps, in my opinion, to illustrate the mysterious feel of the scene, as it looks a bit otherworldly. This very diffuse light probably comes from a very early, possibly pre-sunrise lighting, that is heavily scattered throughout the thick fog. In the beginning, I did the mistake of tinting the light too much into the color I wanted the scene to have, but this resulted in a very monochromatic image, as all the other colors got lost. The right way of doing it was to mainly tint the fog and give the lights just a soft color. Therefore, some of the warmer colors like in the sand were preserved and offered a soft contrast to the dominating blue. The red light of the server tower was made simply with a mask for the emission multiplied by color and strength.
It was also important for the lighting to set the scene’s exposure right from the beginning. When I start a new scene, I usually deactivate some of Unreal’s post effects and fixed the auto exposure. With that, I can view the lighting without the exposure changing based on where I look at. It is also a good start if you are not very familiar with how cameras work and want to gain some knowledge. The Unreal documentation and the internet, in general, have lots of information about this topic, and it is very helpful to understand how images are handled in Unreal to create the results you want to achieve. Until the end, I did nearly no color grading, as I believe that the scene must look at least decent by itself before playing with the colors. I mostly tried to lighten the shadows, so that the information would not get lost within them. I tried to carefully push some reds and blues to strengthen the contrast between them. I also completely desaturated the shadows.
Challenges
The biggest challenge for me was definitely the ocean, as the work in Houdini was very new and demanding. I really like what I was able to learn but must still admit that the whole process of simulating, waiting for results, and tweaking things was very tiring for me as a real-time/game person who is used to WYSIWYG workflows. Nevertheless, I will probably explore Houdini further in the future for other purposes like FX or procedural modeling in combination with Unreal.
Another big challenge was the modeling of the car, as it was a complex model with many features. Here, it was most important not to get lost and always check how certain elements look in relation to the rest of the car. It was also essential to keep the scene organized as I worked with many objects and multiple versions of them. There are many helpful for that matter features in 3ds Max like the aforementioned referencing, plus stacking of modifiers, use of layers and sharing of uniform modifiers like turbo smooth or symmetry between the objects.
Another challenging thing for me was that I put a lot of time into the scene and had little breaks during the phases only when I was writing my thesis. Keeping my eyes peeled and my artistic senses fresh was very hard. Always get feedback from people who are more experienced or whose judgment you trust. Listening to other people’s thoughts enabled me to get so much more out of the scene. Additionally, the feedback was not only important for me to find the flaws in the scene, but also to keep me motivated. I learned what was already cool in the scene and what worked. Maintaining fun at work can sometimes be hard when things don’t work as they should, but for me, it’s important and allows to achieve good results.
Afterword
I hope these thoughts and experience will help or inspire you for your work. If you are interested in my other projects or want to contact me, have a look at my website or Artstation. Thank you for your time!