RTX Nature Lighting and Vegetation Studies in UE4

Note: The Tropical Jungle scene demonstrated in the article is only part of the series of test scenes, the previous works can be found on Finn's ArtStation page.

Introduction

Greetings fellow artists and digital art enthusiasts!

My name is Finn Meinert Matthiesen and I'm a 3D Environment Artist from Germany. I have been working in the games industry for almost eleven years, currently employed as Senior 3D Concept Artist at Karakter Design Studio in Berlin, responsible for the environment concept art development of an unannounced project.

At Karakter Design Studio, we currently establish a state of the real-time concept production pipeline to help our clients with solutions that are way closer to the final in-game or in-engine footage and can be directly used by their in-house artists within their 3D engine application as reference for further content creation.

The studio is relatively popular and has won several Emmy Awards for the art direction of the Game of Thrones TV series as well as provided concept art and art consulting for various AAA games and game series such as Horizon Zero Dawn, Call of Duty, Killzone, or Anno.

Career

Before I joined Karakter Design, I had been Principal Environment Artist at Crytek, working on various AAA and AA titles like Ryse - Son of Rome, Warface, Robinson - The Journey, Crysis, or The Climb. I started there as a young 3D Artist in my mid-20s, just after my graduation in traditional communications and graphics design.

Between my jobs at Crytek and Karakter Design, I also worked as a 3D Lead Environment Artist in indie games, which unfortunately turned out to be the most difficult, disappointing, and exhausting part of my career as an artist in games so far.

Even when I was a young kid, I already had an intrinsic passion for art and games, so I got in touch with 3D quite early and created creative content for games as soon as I could handle a keyboard and a mouse. Early art-driven games like Monkey Island, Another World, or Syndicate had an immense impact on my fascination for the beauty of virtual worlds and environments, and I soon started with creating custom maps for games like Doom, Duke Nukem 3D, Shadow Warrior, or Quake, especially when modding became really big after the release of games like Unreal and Half-Life. It's nice to see that even after all these years, these titles are still well-known topics nowadays and it feels great to have been around as a teenager when those titles started to influence the entire industry. Back then, I typically used the usual 3D editors like Worldcraft/Hammer or UnrealEd, so it took a few years until I wrapped up first 3D assets with software such as 3ds Max, Mudbox or ZBrush.

Since Germany lacked quite a lot in the game development field (and it does until now, even though it is supposed to be the most substantial market for games in Europe as far as I know), I studied traditional graphics design instead of game design. Fortunately, after releasing some screenshots for a fantasy scene I did in CryEngine 2 during my diploma, I received a job offer from Crytek - and that's how I eventually became an artist in the games biz.

RTX Nature Lighting Studies: Motivation & Motives

So in this article, I'd like to explain my workflow, motivation, and motives considering my recent Tropical Jungle study in Unreal Engine 4, which I just posted on Artstation. It is actually the fourth nature study I did in the past months and my working philosophy for this approach was quite similar to the other ones - which I'll explain in-depth later on.

One of the biggest factors, that kept me pushing for these (and further) 3D real-time art studies is certainly the frustrating stagnation as an artist during my time as an indie developer when I essentially lost a few years of my career and didn't really improve or grow as an artist. Plus, fighting my way back into the AAA industry was quite tough as well. I definitely don't want to lament about this time at all, especially since it was my personal decision to take the risk and go for that indie path, but from an artistic point of view, I felt the desire to get back on track with the current or next-gen environment art requirements.

And this sheer desire for art is also the most important trigger and motivation anyway. I was driven by the passion for real-time 3D art in general and the wish to gather more knowledge and experience through various art studies in order to come up with more complex, demanding, and challenging scenes. This is also quite exciting with Unreal Engine 5 showing up at the horizon as well.

Workflow Requirements & Target

In my former studies, I focused more on winter environments or less vegetation based scenes since I still had technical issues with Unreal's RTX ray-tracing features and foliage heavy assets back then. But when the engine versions improved and I did further research on this topic and potential solutions, I finally found quite a few ways to prevent or overcome these earlier issues. This has been a vast eureka moment for me, especially since I always loved to work on veggie compositions back at Crytek and CryEngine times. It clearly took a while for Unreal 4 to match up with CryEngine in this matter, though.

However, my agenda for this study was once more to keep the workload manageable and to focus on the overall mood and big picture rather than on detailed or overly complex assets in terms of creation, especially since time is still the limiting factor with a family and a full-time job. I created and textured so many 3D assets during my career that it wasn't extremely important for me to practice my hard-surface modeling skills, even though I'm certainly doing this in further studies again. So in order to achieve this, I used a lot of my own assets and materials from the earlier projects or the ones I had lying around (without harming any copyrights, of course). I also mixed those up with some ground assets from Megascans, a few bushes from UE4 Marketplace, and some heavily modified and optimized high poly assets from Evermotion. I once wrote in a former article for another media platform that creating a huge set of quality plants takes a lot of time, even with great tools like SpeedTree, which is also the reason why I went for a shortcut on that part (I'll still show some of my own assets later on in order to explain my asset and material workflow).

Apart from that, I started to work on a big foliage and vegetation set for the marketplace, together with my good friend and long-time coworker Armin Chaudhry. But in this case, my main goal was merely to explore the possibilities of nature compositions and satisfying lighting in Unreal Engine instead of pure asset crafting.

Inspiration & Execution

Similar to my former studies, I brought myself into the mood for the scene by collecting and gathering inspiration material and photo reference images of various tropical jungle locations. I also made the decision to go for a jungle setting mainly because I wanted variety in my nature scenes - and after two winter-themed artworks, I was totally in the mood to go back to my Crytek roots somehow, even though rain forest maps are way more common in games. However, apart from a really small CryEngine study and a tech demo vista, I never really showed a typical jungle environment in my portfolio, so maybe it was about time for something like that as well.

While my earlier UE4 nature studies were more experimental and improvised, I actually planned this approach more carefully. Of course, as most artists out there, I had significant doubts if I would be able to achieve my targets and ideas for this project, but still, my goal was to come up with a forest scene that looked rich and satisfying enough - compared to my former nature artworks.

I started with arranging rock and small vegetation chunks to get a feeling for how the materials and lighting would respond and behave (during my career, I previsualized and whiteboxed a lot of environments roughly in 3D before the actual modeling and polishing process kicked in). A common technique I use for such scenes is to import photo reference image planes in order to stick to the direction I wanted to go with proportions and materials and to control the initial lighting mood somehow. They also help immensely in vegetation heavy scenes to keep up the illusion of a dense forest background, making dressing and asset checking way easier.

Scene Management & Setup

Naturally, once the scene becomes more populated with assets, you always end up tweaking it at various spots. I did the same, especially since the two most important key factors were lighting and foliage reaction. And as you can read in a lot of other articles or interviews by lighting artists these days, even though UE4 provides very strong lighting and post-processing solutions, it's always an immense process of trial and error plus it requires tricks and techniques to get the lighting requirements right for the demands of a certain scene. I'm pretty sure that this will change towards an automated process or pre-setup with Unreal Engine 5 next year, where the initial lighting might always look photoreal (UE5 Lumen could be the premiss here), but at the moment it still requires a few unique solutions to craft a satisfying lighting mood. For example, I wasn't able to just convert the lighting from my former projects to this scene, since it simply didn't fit or behaved in a proper way. So for my jungle location, I created a setup that was based on a baked neutral HDRi skylight and combined this with two directional lights - one for the direct sun and with RTX shadows on and the other one as an indirect light amplifier to get this washed out reflective touch on all the leaves. From a technical point of view, the overall lighting and post-processing setup was way more conservative compared to my former scenes. I even didn't use a lot of post-processing features at all, reduced bloom and lens flares to a minimum and only relied on a moderate value for Unreal's SSGI. Even fog settings are very subtle and mainly affect the background.

In terms of composition and asset quality, it was important for me to find the right balance between lush and detailed vegetation assets and avoid noise or loss of readability. To achieve this, I tried to be careful with my primary, secondary, and tertiary shapes when shaping up my main composition clusters and negative spaces where the eye could have a rest. Obviously, in reality, elements don't follow artistic composition patterns and often behave sort of randomly - or at least based on the weird, pretended chaotic rules of nature itself. So compared to movie frames, virtual real-time environments that can be explored from various paths and angles, need a bit more care and work from different spots to make the scene composition, so you constantly end up checking shapes and assets against each other.

And last but not least, materials and lighting play a huge role when it comes to readability as well. The more readable lighting and materials are the more details and haptic patterns you can often add to the scene without screwing it up in a noise festival. That's why I kept my color palette quite moderate and only used a spectrum of green and brown tones for the vegetation while keeping the negative spaces mostly greyish or desaturated for the ground and sky parts. I thought about including some complementary elements by placing some red-colored plants or assets but somehow ended up not going for that approach, mainly due to my choice for the final color grading.

Assets, Textures & Materials

When it comes to materials, textures, or the vegetation shader, there's no real voodoo in there.

When using RTX and foliage, it's necessary to turn on the "Evaluate World Position Offset" flag in order to avoid shading artifacts on the backsides of your leaves. I totally admit that it took me a while until I figured that one out. Apart from that, materials and textures for the leaves and the trunk areas are really basic. For foliage, I used Unreal's classic double-sided foliage shader with a few modifiers to control subsurface color and lighting or to boost and vary albedo color or specular and roughness settings. Proper normals or detailed models with proper tangent and smoothing information (on those plants) were way more important for the overall appearance in the end.

An important decision I made was to go for more expensive tree assets from classical arch viz sources instead of using common mid poly SpeedTree ones with simpler leaf planes. Being inspired by the Nanite feature approach in Unreal Engine 5, I was curious about how far I could get with these true high poly assets that consist of single leaf elements instead of reduced planes. Of course, SpeedTree offers some great techniques to create in-game optimized assets, such as vertex AO baking for example, but in my case, I wanted to work with vegetation that is much closer to reality in its spatial behavior and appearance. Still, I had to tweak the materials and textures of all assets constantly against an evolving lighting setup. I only used Substance for some of my older assets that I put into the scene, so I didn't have to produce that many new textures and materials. For studies like this, you can get away with a lot of rough texture and material work, so it's often not necessary to come up with super complex Substance materials or to spend days in Designer.

While I spent so much time on embedding rivers and riverbeds when working on games at Crytek (The Climb for example), I didn't put any real effort into the river stream here either. I just placed some rough smoke particles and moving decals there in order to tease it a bit. Lazy again, I confess. At least I tried to embed the lower water area with a few moss and tint decals to give it some sort of depth, but also in a subtle way in order not to make those spaces too noisy.

Polish & Further Scenes

Regarding optimization and polish, I didn't put much time into LODs or performance tasks, so there's still a lot of room for that. I might do a few variations of the scene with different lighting and weather setups later on, - and I'm planning to revisit some of my older UE4 studies in order to improve them a bit, although I'm already working on something new (next to the market store project I mentioned above). So let's see where the creative flow will push me in the near future.

General Tips and Techniques for Real-Time Scenes

Finally, I'd like to mention a few general pieces of advice for creating realistic scenes in real-time and I hope that this information might be valuable for some of you, at least for beginners.

Before going too crazy with putting tons of market store assets into the first startup scene, make sure to become more familiar with the engine itself. Assets and materials often appear, react, and respond quite uniquely in different engines, especially when interacting with lighting. I highly recommend studying the individual features of a specific engine before starting with asset production and also trying to differentiate bad quality assets from good ones.

It's also a reasonable and very useful step to learn from existing scenes of higher quality that were made for the engine of your choice. Most popular engines out there, like Unreal or Unity, provide tons of free (or affordable) example scenes in their market places, so taking a closer look at how they work and how all elements of those scenes interact with each other is certainly the best way to start in order to understand how assets, materials, and lighting behave and how to achieve various effects. In fact, I did (and still do) the same if I stumbled upon something that really looked promising. Observe, mimic, and adapt processes that will help you on your way.

Additionally, photo or movie references also help tremendously on the way to achieve a realistic look. And once you have been able to establish a promising mood in your scene, try to use that work as a blueprint for your next project. Keep an eye on your pace and don't start dressing with too many unoptimized or too different assets. Slowly populate your scene with elements and take your time to reflect whether you're still satisfied with the overall look.

Apart from assets, materials, textures or lighting, scene composition is another major key when it comes to realistic-looking environments, but don't stress yourself too much with the elite theory of the good old masters of composition. Try to have fun with your work on the scene. Trust me, your perception and intuition for composition will evolve over time, so it's normal if you struggle there as a beginner.

Thoughts about the Future of Production

With Unreal Engine 5 showing up at the horizon, a lot of artists seem to perceive it as a game-changer in production and while I'm usually on the skeptical side when it comes to tech demos, I admit that features like Nanite and Lumen have the potential to be revolutionary.

As usual, there are risks and chances connected with such a fundamental pipeline change. Game developers and Game/3D artists are always aware that they constantly have to adapt to new technologies and crucial pipeline improvements and of course, a lot of the expertise artists need these days (performance optimization, LODs, proper normal map workflow, etc.) might be obsolete in a few years. Even today, we can observe that 3D artists from the areas of arch viz and the movie industry start to use game engines for prototyping and visualization, so while game artists and offline rendering artists were sort of separated camps a decade ago, they moved much closer to each other in the past years.

I can also imagine that AI and automation tools will have a huge impact on the entire production of 3D, at least when it comes to scenes and environments that consist of generic elements, like nature scenes for example. Procedural landscape and vegetation scattering tools are already in use these days, especially when creating the base for huge open-world environments. Combined with a strong base library (like Quixels Megascans or SpeedTree) and an optimized lighting setup from startup, I'm pretty sure that it won't be much of a challenge to come up with cinematic or photoreal landscape scenes in the near future. Programs like VUE provided offline rendering solutions for landscapes and nature scenes even years ago and there are quite a few groups and companies working on projects for Unity and Unreal that focus on exactly these aspects, Epic/Quixel, SpeedTree or Substance being probably the most prominent ones to mention at this point.

Nevertheless, a lot of creative tasks will rely on 3D artists. And while more generic environments might be easier to achieve, artists still have to craft unique sets for architecture-based solutions relying on their own creative inspiration. AI recombination and scattering tools, procedural landscape generation or advanced photogrammetry workflows might greatly improve the quality in certain artistic production areas, but stylized, specific, iconic or unique geometry, like fantasy or sci-fi architecture and designs, will need to be handcrafted for quite a while I think.

Afterwords

Well, these are my rough assumptions about what's going to happen at some point. Pretty sure that the future will prove me wrong on a lot of aspects like it always does somehow.

I sincerely hope that this insight was at least a bit helpful and interesting for you, so thanks a lot for reading and all the best with your creative projects. And now, go and craft some impressive stuff!

Cheers!

Finn Meinert Matthiesen, Senior 3D Concept Artist at Karakter Design Studio

Interview conducted by Arti Sergeev