How to Set Up Photorealistic 3D Environments For a Game in UE5

Introduction

I'm Ali, an artist from Turkey who's currently living in Portugal. I've always been into art and from an early age I was busy making doodles and creating playable worlds on paper for my friends. Some of these designs were quite successful and got my whole class hooked on them. Still to this day, that feeling of "designing worlds" is what I'm pretty much addicted to.

Back then, what inspired me was mostly comics and games. I grew up with Tintin, Asterix, Spirou, Lucky Luke, and countless other European comics, which were quite easy to find in Turkey. Later, that developed into more mature stuff, like Heavy Metal comics, and the works of Jean Giraud, Milo Manara, François Schuiten, etc. Also, I've always been into computers, playing games, learning weird tools to make stick fight animations, creating basic text games, and even some online text-based RPGs played on old forums, which I'd design and act as a game master. 

After I started considering art as a serious career, I only focused on drawing and painting, as I wanted to become a Concept Artist/Animator. I even fantasized about becoming a fine artist at some point, but the strong attraction of games as a medium always kept me coming back. I think it's the most interesting and potent of all art mediums and still has so much to explore.

Back then, I really hated 3D. I thought it was a soulless hack that only fake artists would employ. I was one of those pretentious purists. But then, the idea of creating our own games with my friends popped up and I didn't want to be bound to only doing 2D games. I started learning how to use 3D and make stuff with it. I completely fell in love with it. I think the way it allows me to compartmentalize the image-making process into very technical steps and focus on each element one at a time, really fits the way my brain works. So I kept practicing 3D and learning about the game development pipeline while working as a 2D artist.

Looking back, I think that my purist approach did me well, in the sense that it forced me to obsess over the fundamentals of drawing and painting. If I had the shortcuts of 3D back then, I probably would have been a less capable artist, though I think I still lack so many fundamental skills, especially with painting. However, every skill I honed with drawing practice still serves me really well while making games.

In terms of projects, I've started my career as an illustrator/concept artist working on the game Mount & Blade: Bannerlord. It was the only large-scale game project in Turkey and honestly, it was the perfect place to get my feet wet, I learned so much stuff there.

In addition to that, I've contributed to a handful of indie games as a freelance artist, more recently – We Kill Monsters and Gecko Gods.

The Team Behind Rytma

The team behind Rytma is our tight-knit group of childhood friends. We've been close friends for more than 15 years and we have always been artistic-minded, though our domains of interest differed quite a lot as the years went by. Eren and Baris are jazz musicians, and Baris specifically has been playing and composing professionally for years now. Eren is more technical-minded and this led him to pursue sound design, production, music technology, and programming in an academic career. So our share of responsibilities divides itself pretty naturally: I do everything visual, Eren does the programming and sound design, and Baris does the composing. We do the game design together, as we almost always share the same frequency in terms of what sort of work we want to produce. As the initial idea of Rytma and the world/setting it takes place in originated from me, I do the narrative/writing part as well. Also, we've gotten additional help from another friend called Kashif for animation programming, which has been incredibly valuable.

The Idea of Rytma

The idea of the game came pretty spontaneously while I was with Eren and Baris. We were roommates for a while. One day they were practicing their instruments in a jam session and I was just sketching next to them. The way they played with each other and improvisationally came up with new music that fit the accompanying music, and together developed something unique and different each time was really fun to witness. At the time, I was also replaying a classic game from my childhood called Patapon, which is this unique rhythm-god game, one of my favorite games of all time. That moment naturally led me to think about music in games, more specifically music-making as gameplay, and how it seemed really unexplored as a concept. The seeds of that moment developed through the years into more concrete ideas and later, more complete gameplay loops that we kept prototyping and shaping. 

In general, music has been extremely inspiring for almost every aspect of the game, possibly more than other games that I love. Not only in defining the mood that we're after but also for gameplay ideas, world and culture details, narrative tone, design philosophy, etc.

Some of the other games that shaped my vision and the way I approach designing Rytma were The Witness, its incredible constant non-verbal communication with the player, and the way it develops simple ideas to their maximum extent; Outer Wilds, its sense of mystery and secret-keeping, gating player access to hidden dimensions just by means of player knowledge, instead of arbitrary metrics like levels or stats; Shadow of Colossus, its gorgeous atmosphere, air of mystery, and its obscurity and simplicity in its design.

In terms of plot, Rytma isn't the type of game that aims at telling a story, and there are no heavy narrative elements for this reason. Instead, it's a game that's trying to explore musical and audial ideas in a thematic context that fits this pursuit.

Setting Up Rytma's Environments

For the world of Rytma, I had a pretty clear vision of what I wanted to do. It was purely inspired by the Anatolian and Mediterranean parts of Turkey, where I've spent all of my summers since I was a child. To me, the Mediterranean atmosphere is just unmatched in its beauty and homeliness and it was a perfect fit for such a personal project. It also fits quite nicely with the themes and more esoteric ideas that are explored in the game. Though I didn't want to recreate it one-on-one, I've mixed a lot of other inspirations from more Eastern origins, especially in the style of architecture, ornaments, garments, and motifs.

I'm not a big fan of procedural content used in projects that aren't specifically designed for procedural interaction, so the environments are all handmade, and this forces us to keep our scope smaller and our focus more clear, which I think is good. However, I have to say I use a lot of third-party content, like assets from the marketplace, for example, Fluid Ninja or Quixel Megascans. Some of this stuff is just way beyond my capabilities so I'm trying to make up for it as much as I can. Though I always try to modify and personalize every asset that wasn't made by me, because that is crucial to fit them seamlessly and cohesively with the rest of the world. Otherwise, you could easily get the despised "asset flip" look, which a lot of projects seem to suffer from, even triple-A.

My workflow is pretty simple, though somewhat novel – since the recent developments in Unreal Engine have forced quite a change in the way I make the assets. I use Blender as a hub, so the work for every environment asset, character model, foliage, prop, or animation starts there. 

For the environment assets, I create a simple blockout in Blender and take that to ZBrush to sculpt further. There I mostly focus on secondary and tertiary details and decimate the mesh when I'm done with it. We all know that Nanite is this crazy paradigm shift in terms of polygon budgets, but still, I see no point in making assets with millions of polys just because you can. It's just inefficient and doesn't look any better. As far as I remember, none of the assets I've made so far exceeds the 300k mark, which is still insanely high, and on average I'd say they are below 100k. I find that to be enough to look good without any normal maps, for a style that is seeking realistic detail.

After sculpting, I use Substance 3D Painter to texture the assets, and another unconventional thing I do with texturing is that I treat around 80% of the assets as hero assets, meaning I create handpainted unique Texture Maps for each mesh, instead of using standard textures that are tiled with some masks. This method has both positive and negative effects on the workflow and the final result. Negative effects for the end result are that it's more demanding performance-wise, especially on texture memory and the number of texture calls required. However, the streaming virtual textures feature in Unreal mitigates that to some degree. I have found it to be mostly fine for the current generation. Also, this method allows me to work with just a single low-mid resolution RGBA texture for each asset, which might result in some performance gains compared to the standard tiling texture method. Workflow-wise, it just takes so much more time to paint unique textures for each asset, instead of preparing a handful of tiling textures, but that's one compromise I'm willing to make for the desired end result I want. The positive effects are pretty obvious. I think it looks better. I have so much more control for each asset and I can work on each one of them specifically. I also think it's pretty enjoyable from a workflow perspective. It also compensates for the fact that Nanite doesn't allow for per-instance vertex painting, which was a great way to add uniqueness to each specific object in the world.

In the end, I don't think the technical aspects of my workflow matter too much in relation to the end result. I believe what truly matters is maintaining a cohesive and clear visual identity for a specific project. It is essential to limit your sources of reference and inspiration and continually explore new ideas that have not been extensively explored before. Instead of constantly iterating on the same themes like "Ghibli-inspired cozy environment" or "Bleak sci-fi corridor," it is important to push the boundaries and try something different.

By the way, if you want to learn more about my work in Substance 3D Painter, you can check out my Twitter where I gave a lot of details on my workflow.

Rytma's Gameplay Mechanics

Rytma is an experimental game solely focused on musical interactions, understanding obscure audial puzzles, and internalizing these new audial ideas to perform musical sections. It involves encountering ancient instruments and deciphering them by playing, discovering their limitations and qualities, learning the "intended way of playing" each instrument through these audial puzzles and guiding elements in the environment, mastering them through play - and finally performing a section of music to prove their understanding, to progress further across the islands. Each instrument is also deeply tied to an environmental element, like the wind so that it changes the way it behaves and the way it sounds – as these elements in the environment change. 

Approaching the Business Side of Things and Promoting Rytma

We basically don't have that sort of concern at this point. We just work on the game to make it as good as we can make it, and I sometimes post stuff on Twitter, even though I don't like doing that too. Ideally, we want to work with a great publisher that understands us, so that we don't have to deal with these things at all. Personally, anything I do that shifts my perception of this project from an "experiment/art project" to a "product" makes me despise the process and I usually try to avoid it if I have the luxury to do so.

Speaking of the Current Plans

The audience should have no expectations, but if they are intrigued by what they see and hear, they are welcome to follow the progress and potentially purchase the game upon its release. We are still in the early stages of development, with plenty of work and exploration ahead for this project. In the meantime, they can follow me and Eren on Twitter for occasional updates on our progress.

Thank you for reading the article!

Ali Eser, 3D Artist

Interview conducted by Theodore McKenzie