Neon Sins: Creating a Realistic Cyberpunk City in Unreal Engine

Introduction

Hello! My name is Indrajeet Sisodiya. I go by Indy. I’m working in Pixomondo, Toronto as a Senior Compositor/Environment Artist. I also teach Compositing from time to time at Seneca College. As a Compositor, I have delivered high-quality photo-realistic visuals for TV and films like The Mandalorian, Star Trek Discovery, The Umbrella Academy, Midway, Fast and Furious, etc. Recently I was selected for the Unreal Engine Fellowship program at Epic Games after which I joined the Unreal team here in Pixomondo to leverage the power of real-time rendering at cinematic levels. I’m very excited as we are working on one of the biggest LED wall stages in the world, here in Toronto to channel the power of Virtual Production in upcoming movies, TV shows, and commercials. But that wasn’t the only reason I decided to dive into Unreal Engine.

I have always been this kid who used to daydream a lot and had these grand visuals in my head that I wanted to bring to life. I would do that as a child with drawings, later with old-school 2D animation software such as Adobe Flash 9 and eventually making my way to doing my bachelors in animation and VFX in India where I learned Autodesk Maya, Adobe Photoshop and After Effects. Then in 2015, I learned Foundry’s Nuke in Seneca College, Toronto. Although, ever since I started working in the visual effects industry, I didn’t really find much time to learn new tools. However, in 2019, I bought Redshift and started looking into the power of GPU-based rendering. Redshift was fast at rendering but it wasn’t real-time.

The pandemic allowed me to have more free time at home and I decided to pick up a software to learn. This is when Unreal Engine caught my eye. Their work has predominantly been in games but slowly they are catering more to the production studios out there. With the recent acquisition of Quixel and Megascans, on top of the already thousands of high-quality assets that were on UE’s marketplace, I was totally sold into learning this. I picked up the courses over at Unreal Academy, UE’s educational portal for anyone wanting to learn the software. I highly recommend it as the courses are very neatly arranged, cover a variety of topics and best of all, free for everyone! I kept toying around with little concepts until I felt I knew enough to do a bigger-scale project in UE.

KitBash3D Contest

KitBash3D is an amazing website that offers modular 3D kits ranging from post-apocalyptic to futuristic, modern to ancient cities, huge worlds to tiny props, and much more. They frequently host a contest called the KitBash3D Contest where they offer a free mini kit for anyone who wants to participate. The goal of the KB3DContest is to create an image or a video based on the theme of the contest. This time, the theme was “Neon Nights”. Some of the key lines from the theme descriptions that really resonated with me were:

• The idea that time is of no essence in this city. Why would you care about the time when the sun has long since been replaced by the sheer power of Neo City?
• The idea that you dangle off the edge and watch as the world vibrates below your feet.
• The idea that the night thrusts us into an all too familiar neon blur, void of cordiality, taboo, and dreams of not-so-distant futures.

Before this contest, I had participated in their last contest whose theme was “Cyberpunk” and my entry called “Tears in the Rain” was a very dramatic Cyberpunk world akin to Blade Runner 2049. This however was all done in Redshift and took me roughly 2 months to wrap it all up.

Having done this prior, this time I wanted to create something different. I focused on the idea that the contest was more about a “Neo-City” rather than a traditional cyberpunk/futuristic city. Neo city simply meant a new city from the near future which made me ground this more into reality.

Although I drew inspirations from the cyberpunk movies like adding the floating futuristic advertisements, the classic dancing ballerina hologram from the Bladerunner 2049, and many more, the biggest inspiration came from the prompts themselves that I mentioned above. I wanted it to be a city that was visually more believable. I knew if I added any crazy architecture or buildings to the city, it would make it less realistic. Same with technology, I wanted to create a world where technology was similar to ours and that’s why you don’t see any flying cars or any cybernetically augmented humans. I wanted to deviate away from the classic cyberpunk genre media such as Ghost in the Shell or Akira while still basing the genre in reality.

But like everyone else, it all starts with an empty slate. 

Opening the Kit

The very first thing I did was download the NeoCity kit that KitBash3D gave for free for this project. Luckily KitBash3D allows artists to download the UE project file from their website for their latest kits. This saves a ton of time as a couple of years ago, we had to bring in FBX files from KitBash3D’s website and had to manually bring in all textures and recreate every shader for all the assets in UE. This took so long and was painstakingly slow. Luckily these new kits are preconfigured to work flawlessly in UE.

Upon opening the UEProject file that came with the kit, I was surprised to see there was nothing in it. This is understandable as the buildings haven’t been loaded into your level as you can see your viewport and world outliner panels only have the basic things.

If you go in the KB3D folder in your Content Browser, you will find four subfolders called Actors, Geometries, Materials, and Textures. This is how your building assets are broken down and collected by the type of asset. All textures that go on the buildings would be in the Textures folder, all the shaders in the Materials folder, all the 3D models in Geometries, and finally all of this combined into a Blueprint in the Actors folder.

For the beginners out there, for the purpose of this tutorial think of Blueprints (BP) as Smart Groups where you can combine assets together and place these BP in your level. So next time you want to change the texture of a sign on one building and that building has been duplicated and placed 20 times in the level, you won’t have to change it for every instance. You simply change it in the BP and that change would propagate to all instances.

So then I simply dragged all the buildings BP in the Actors folder into my viewport and spaced them out. You can see there were a total of 9 pieces, 3 large buildings, 3 medium buildings, and 3 tiny pieces. Since the 3 tiny pieces are only close-up small props, I basically had 6 buildings to make an entire city without it looking too repetitive.

Challenge accepted!

Scattering in UE

Now from here, there are two ways of spreading these buildings out. One is the harder but more unique way which is to hand place every building around and make the whole city this way. This method is usually good for doing shots that are no wider than a couple of city blocks. However if your shot involves much wider establishing shots similar to mine, I would recommend automating this process in some way. So I decided to go down the route of turning the buildings into the foliage. This is where UE blew my mind with its amazing performance.

Foliage Tool: UE has a powerful painting tool that allows you to paint an object or a bunch of objects on the surface of your geometry. It is specifically used for (as the name suggests) painting foliage such as grass, rocks, trees, debris, and such on your landscape. This is pretty much what I had used it before for, scattering tiny to medium-sized objects with fairly low poly counts. I was very cautious this time as my plan involved turning these massive high poly buildings into foliage actors and painting them like one would paint grass.

To turn the building BP into foliage actors:

1) Go to the top of and change the “Modes” to “Foliage”

2) In the Foliage Panel, click “Add Foliage Type” and select “Actor Foliage”. In the following pop up box, save the Foliage file with a good naming prefix like F_(Name of Building BP)

3) With the new Foliage Actor created, select your corresponding Building BP and drag and drop it over the Foliage Actor icon. Then select the Foliage Actor and click on Actor class and search “KB3D” to see the list of our BPs. Select the same BP that we just dragged over. Your thumbnail should update to the corresponding BP.

Repeat all these steps with all the 9 props and now we are ready to paint!

4) Now in order to paint, I changed the “Modes” back to “Select” and created a plane at 0,0,0 location. This would be our city ground. I scaled it up to 1000,500,1 and switched the “Modes” back to “Foliage”.

5) Then I checked the little checkmark on each of my Foliage Actor icons except the 3 small props. Checking and unchecking this box means these selected assets would be scattered when you paint. It is a quick little way to only paint selected assets instead of painting everything.

6) Next, change the brush size on the toolbar to 8192 so it is a large paint area. Then shift select the 6 building Foliage Actors and change the density for all of them to 0.01. The density slider in each Foliage Actor is a great way to paint more/less specific buildings. Higher density = lots of instances of that actor, lesser density = only a few instances of that actor get painted.

Also, change the Radius on all of them to 5000. This means different instances won’t be packed next to each other but at a given distance away. 

Usually, the default value works great as the use case for Foliage Tool is painting grass and rocks which are tiny and can be densely packed. In our case, since these buildings are huge, we just need to paint a few per stroke.

And just like that, we painted our first stroke, which by the way is 10 Million polygons being drawn, lit, and rendered at 120 FPS! When I did this for the first time it blew my mind as UE didn’t even break a sweat. Now go ahead and start painting your city blocks. 

You can also adjust the Min and Max scale values on each Foliage Actor to paint them at different sizes. This is a great way to break up uniformity as all the tallest/smallest buildings don’t look alike. Be careful as to not make a building too stretched or squashed. The human eye can quickly see a reference point such as a window and tell when it's out of proportion.

One big feature of the foliage tool which I wish it had is the ability to place instances at specific angles to each other. This would allow the buildings to not be rotated randomly to each other but follow some sort of order. This is an important thing to keep in mind. Look at references of real city blocks and see how buildings are arranged. In certain cities for example Toronto, most city blocks are grid-based so the buildings would always be 90 degrees to one another. However, cities like Hong Kong have had a sporadic growth in buildings that do align to one another but overall are random compared to neighboring city blocks.

7) Use the select tool in the Foliage Painting Toolbar to select one or many buildings at a time and rearrange them by hand to create some order in chaos. Also, scale them individually to different sizes to break up the repetition. Here’s the result after a quick 5-minute tweak. The longer you “city plan” the more real the city would be.

Laying the Groundwork

The first few paint overs scattering the buildings were initially done on a huge flat plane similar to how I showed it above. But it didn’t look as dramatic. Then after re-reading the prompts from KitBash3D I realized that in order to show a city where the night starts even when the sun is still setting, I thought what if I raised the ground that the city was built on? This resulted in the following setup where there is the lower city, then a slope that leads to the higher city.

This also visually separates the classic "elite city" from "grungy-crime-infested city". If you are creating your own city, try to lay down the groundwork before you commit to spreading the buildings around.

Texturing

The buildings that come in Kitbash3D kit are already pre-textured which is nice, however, the biggest change I had to do was the windows. As you can see below, the material of the windows is a simple translucent reflective window. The goal was to create a window that would be opaque and emissive similar to windows at night.

The biggest challenge was to make it random enough where people wouldn’t be able to see repetition in similar building windows nearby.

I grabbed half a dozen window textures, some monotone, and some colorful neon images of cities like Hong Kong and Tokyo that I found on textures.com under the building category. This basically drives every single texture of my project from windows to ground textures. For the sake of simplicity, I will just call them window textures.

Here is the master material for the windows that I created. It might look like a lot of nodes but most of them are very simple. Think of it this way, more controls you add in your master materials in the beginning, more variations you can create later. I will take you through the setup, however, please do take it with a grain of salt as the shader was made in my early UE days so I am aware certain things might be done incorrectly or inefficiently. Having said that, let's dive in!

At the very top, we have the Base Texture section which is a Texture Parameter node in which the user can decide to input his own texture, in my case, the window textures. On the left, we have simple UV controls. Texture Coordinate node gets multiplied by a master texture tiling amount (which is a constant you create and then right-click and convert it into a parameter). You make two more such constants and these become your individual U and V controls to determine how much to tile a texture in either direction. 

Then we do similar setups for three more sections called Hue Noise where a black and white Noise texture goes through the same setup of UV controls. This is the primary driver of the color variation of all the windows. The output of the window texture image from above gets plugged into the Hue Shift node’s Texture input and this Hue Noise gets plugged into the Hue Shift Percentage input. So wherever there is white in the noise will have hue changes, wherever it is black, the original windows texture image passes through.

Below Hue Noise is Hue Mask. This again adds another layer of variance to break up the uniformity even further. I will showcase how it plugs into the rest of the shader after I explain the last section under that which is Brightness Mask.

Brightness Mask is again a noise texture with tiling controls that allows up to decide the brightness of the windows. Wherever there will be white from the noise overlapping the windows textures, that area can be as bright as the user wants and wherever it is black, those remain unchanged.

Finally to showcase how everything plugs in. Below you can see where the outputs are coming from. Base texture goes into Lerp node’s A input, Hue Shift’s result goes into Lerp node’s B input and Hue Mask’s result goes into the Alpha input of the Lerp node. What Lerp basically does is tell UE, hey look at the Alpha Map, wherever there is black (value of 0), show whatever is plugged in A input and wherever there is white (value of 1), show whatever is plugged into the B input. Lerp is a very powerful node that I think all UE artists should use often to blend results like these together.

This result then goes directly into a Desaturation node controlled by a desaturation value that the user can adjust. Finally plugging into the base color of our window geometry. This step is a bit redundant and could be avoided if need be as the windows are anyway going to be driven by Emissive Color more than Base Color but I connected it in just in case. Another copy of our Lerp output goes into contrast to make the window textures punchier which then goes into another global Brightness Intensity control.

Lastly, the output of our Brightness Mask gets clamped between a value of 0 and 1 and then passed into the alpha input of another Lerp. An input of this Lerp has our global Brightness Intensity control’s output and B input of the Lerp is set to 0. This is done to make sure not all windows of the building are on. Depending on the Brightness Mask noise, some will be on and some off. And that’s it, finally, it goes into the same Desaturation controller and gets plugged into the Emissive Color hence driving our entire city’s windows!

All we have to do now is to find the master material that we just made in the Content Browser and right-click it and select Create Material Instance. This will make a copy of the material but expose all the parameters that we have built before. Here is what one of them looks like.

I then plugged in one of the window textures into the texture slot. Then tiled it a lot globally. and adjusted the Hue controls and Brightness controls to create the variations you see. The same image repeats but it has different colors and intensities each time. This brings more realism than simply plugging in the window texture directly onto our window geometry.

Please be aware though, some values I have input here might be too bright or crazy if you try to replicate this in your scene, I set mine so high as my camera exposure is very low as I wanted that nighttime feel. Feel free to tweak the values to what works best in your scene. Also, I would recommend playing around with different input images to see what works best!

Here is what one of my buildings looks like with the texture on. You can see how each window has different hues of colors and how some are really bright (and hence blooming) and others are not emissive at all. This is the most important driving factor to achieve realism with window textures in my opinion. 

Overall my city looks like this so far with the buildings laid out and materials now creating different window lights.

As for the ground texture, I used the same method and repeated one of these window textures a lot, and given enough repetition, it would be very hard to tell what it is. This is a great example of only putting in effort and doing smart work rather than hard work. If we know the camera would only be high up in the sky and never come down, there is no need to detail the ground level.

The next step was to add the much-needed NEONS!

Neon Lights and Holograms

Creating Neon Signs

To create the Neon signs I was initially thinking of going about the same route of finding images of Asian cities and grabbing signages from there. However, while browsing the textures that came with the KitBash3D kit, I noticed this image which immediately made me have a sigh of relief as it was the complete signage image they had made for their Cyberpunk Streets kit. And since few assets from that kit were included in the Neo City pack, this texture was also included. One image to rule them all!

The bigger question now was, how do I incorporate this into the scene. I tried thinking of procedural ways to automate this process and I am sure given enough time I could have come up with a way but time is what I didn’t have as the deadline was coming up in 2 weeks. So I decided to do it manually.

I opened this up in Maya but you can do it in Blender or any 3D application. I created a default square plane with no subdivisions. Then projected the UV from the top so the texture sits perfectly inside the plane as you can see on the left below. Then as laborious as it sounds, using something like Multi-Cut tool or any similar tool in your 3D application that allows you to create edges by clicking on the surface, dividing the whole square into mini polygons, and eventually deleting the empty spaces and separating all the pieces into individual geometries. This method works better than creating new planes and adjusting their UVs for each sign because if your sign is a rectangle but the original geometry you assigned the image to is a square plane, then arbitrarily changing the size of your geometry until the sign “looks” right is hard.

When you use the polygon cutting method, the final size of your sign is predetermined and does not need guesswork.

Once done separating all the pieces don’t forget to center the pivots for all the signs and put them all on 0,0,0 in your scene before importing to UE. This way they will import correctly and won’t show up with an offset. I made a total of 100 signs like this, some from this, few from other images.

Creating and Animating Holograms

Adding the Neon signs to the scene was one-half of the puzzle. I knew if I was going to move my camera around, I can’t exactly have just static signages especially if we are in some sort of future. I wanted to have some animating signages but I knew animating each one of these would take way too much time and I would end up missing the deadline. So I started looking into bringing pre-animated signs of all kinds.

As evident from the aforementioned Tears in the Rain project, I already tackled the issue of animating signs in this, while I was trying to make a cyberpunk city in Maya and render with Redshift. I have always been fascinated by GIFs and how such an integral part they play in shaping our culture. I thought there was no better way to get the hot and happening stuff of our culture from anywhere else other than GIFs. Other than the content variations you find in GIFs, they are also a very lightweight format given the fact they are playing an image sequence which in general are quite resource-intensive. However, the issue is that GIFs aren’t natively supported by 3D applications which is why for Tears in the Rain I had to painstakingly convert every GIF into an image sequence and plug that into my materials. The good thing about this is that I already have a library of all kinds of GIFs that are in GIF or image sequences, whichever the need be.

Just like Maya, UE doesn’t natively support GIFs. Since I still was learning UE, now I started learning how to import videos and image sequences into UE. This took at least 3-4 nights of playing around until I realized the process of bringing in videos/image sequences in UE is unnecessarily complicated using Media Textures and Media Players. There are ways of also bringing in animated media using Sprite Sheets but they too are unnecessarily complicated where you have to turn image sequences into a glued sprite sheet and then animate that using panners.

If I had to bring in even 50 GIFs this way, it would take absolutely forever. Luckily in comes the developer Neil Fang on the UE Marketplace whom I would love to thank from the bottom of my heart for creating a superb GIF Importer for UE and making it as easy as dragging and dropping a GIF directly into your project. His plugin literally allowed me to finish this project on deadline!

So I thought, I was done with the hard part, I just drag and drop all my GIF collections of animating signages. 

But life has a way of throwing a wrench at every given turn. Whatever can go wrong, will go wrong. The moment I brought in a GIF, it crashed UE. I spent the next 3 days just trying to find why it was crashing, given the fact, I already was a beginner in UE and didn’t know much about it.

It could be any of the million sliders and checkmarks UE has that could cause this. Eventually, I realized after a whole bunch of trial and error that the GIF importer he created was incompatible with UE’s Raytracing features which I was using heavily for reflections. Bringing in a GIF while having Default RHI in your project settings set to DirectX 12 would immediately crash it. So now I had a tool but no means of using it. I didn’t want to turn off Raytracing as it was integral to my project. So I reached out to the developer on multiple platforms, eventually submitting a bug on his GitHub explaining the details. This happened a week after I started working on the project. I had exactly 3 weeks from then left for the deadline.

I had my fingers crossed as I didn’t know if he would even see my message and secondly if he does, would he be able to fix it before the deadline? Every day, multiple times a day I would refresh his Github page to see if he responded but nothing. I kept working on other aspects of the city design while waiting for his response. He responded back exactly 5 days before the deadline and luckily he mentioned he had a fix! This meant I had 5 days to set dress the city with signages and get everything out, composite, add sound effects/music, cut, edit and submit the entry while still having a full-time 9 to 6 job.

The reason I am mentioning all the hurdles is to quickly segway into something important when working on projects like this. Life will always throw curveballs at the last second. Never get disheartened by it and keep at it. Give it your 100 percent, the final stretch will always be stressful and mind-numbing but if you make it through, there is nothing but rewards waiting on the other side!

Back to UE!

Scattering the Signs and Holograms

Now with all my assets ingested into UE, I then had to hand-place them throughout the city. While I was awaiting a response from the GIF Importer dev, I started putting all the neon signs on the rooftops. One important thing to mention here that applies to every project you might do, is to always ask the question Why? If you are placing a sign somewhere randomly, ask yourself why is it there? Why would anyone even look at it there? Doing this makes you understand your world better allowing you to strategically place all of them in places that would logically make sense. Put them in places where you think someone would have a billboard space rented for commercials. Cluster them together as in real life advertisements are fighting over billboard real estate. Put them in easy visibility from roads/other vantage points. Doing this also helps you segregate your city into districts. Advertisements with financial content, banks signages, etc could be in your financial district, movie theaters/clubs/bars in the entertainment district, other generic ads in your residential districts, so on and so forth. Being observant and Imitating what you see in real life is the number one factor in creating realistic content.

And now you can see this is what it looks like after adding all the neon signs and animating holograms!

Adding Physical Lights

Until now, all the so-called “light sources” in the city are just emissive neons and holograms. Being emissive, they would be naturally visible in the darkness and reflections however they won’t actually light up surfaces around them. For this, we will need actual lights in the scene.

Below you can see what the scene looked like with just emissive materials as they get reflected into surfaces.

And this is after turning on all the lights that actually light up the scene.

One more example below of the base scene with only emissive materials and no lights.

Now with just some overall large-scale lights on.

And finally with additional detail lighting to complete the scene.

The way I majorly lit the city was using massive rectangle lights (size of couple city blocks) to light the actual buildings and point lights to light little ledges/detail lighting. In order to make my lighting realistic, one big thing I try to do is break up the color of the lights to create more hue/saturation/value variations. Most real-life lights are not single-colored. This is why whenever I create a rectangle light, I set the light texture to an image instead of a single color.

Here is a rectangle light with a color coming from a single orange color swatch.

And here is the same rectangle light with a source texture set to an image of overhead traffic view at night.

The difference is pretty impressive. This is why I would highly recommend lighting with images instead of single colors. Another trick to lighting something big like a cityscape is to follow what I call the “dark on light” technique. A pretty age-old trick that enables you to create a sense of depth by having your foreground buildings be dark and placing lights on buildings directly behind them, making those background buildings really bright which in turn makes our foreground dark building pop more. Layering lights like this throughout the city (keeping in mind the camera’s point of view) will give tremendous depth to a city that otherwise would be very cluttered because of the countless windows.

Last couple of quick tips. If you are placing dozens and dozens of lights and thus hindering performance, I would suggest turning off the “Cast Shadows” option especially if the massive rectangle lights are lighting 3-4 city blocks at a given time. Most lighting from the street is generally diffuse and devoid of harsh shadows. However, don’t overuse this as shadows are very much necessary for good layering of lights. Also keeping your Attenuation Radius (sphere of influence of the light) to something smaller would help you gain back some performance.

As for the Sun Light, I used the Sun and Sky Light under the Lights tab. This blueprint auto-creates a Directional Light (sun), Sky Light (ambient light that captures the sky colors and uses that to light the scene uniformly), and Sky Atmosphere (the component that takes the position of Directional Sun Light and changes colors of the sky accordingly). All I did was put the sun down low to create a nice sunset warm mood.

And this is what it looks like after adding all physical lights to the scene!

Adding Exponential Fog

Fog or atmospherics play a really big part in bringing any wide-open landscape shot to reality. Light doesn’t just light up surfaces but also the volume that it travels in whether it is air, water, or solids (transparent objects).

Little particles of dust suspended in the air catch light and as a whole lift up the dark values of the scene. Unreal Engine’s native Exponential Fog is a great tool to get started with atmospherics however it can need a bit of tweaking to get it working in your scene. The first thing I tweaked was changing Directional Inscattering Color from default to black.

Directional Inscattering simulates fog to be stronger towards the direction of Directional Light (that has atmospherics enabled). Here is the default Exponential Fog with default Directional Inscattering settings.

The fog is pretty uniform everywhere. However, if I set the Directional Inscattering Exponent to something higher, you will see how the fog gets more visible towards my Directional Light.

Now, this might look desirable and is definitely useful in some cases, however, if I move my camera down you will see that fog still lingers in areas closer to my sun direction whereas it should have been blocked by the building’s shadows and should have been stronger higher up in the sky.

To reduce this issue, I set my Directional Inscattering Color to black so that the primary color and density behavior is driven by my Directional Light (sun). Being the fact that it is already sunset, now my color of the Exponential fog automatically matches and shows up orange.

Here’s an example of a scene without and then with fog:

Making the City More Alive With Traffic

By this point, the city was already looking pretty good but any time the camera looked straight down, there were no roads visible with any traffic. Everything seemed detailed but was devoid of life. Now by this point, I barely had days left for the deadline so instead of adding 3D roads with 3D cars (which would take weeks if one wants to achieve real-life-like street details), I decided to use 2D video elements and composite them right inside UE. This is an important thing to remember, a lesson I learned while making this. 

If you are in charge of the final creative look of your project, what matters is the final result. A lot of 3D purists would look at this as cheating but I personally think if your goal is to get from Point A to Point B, then take the wheel that is already invented and reach your destination. There is no need to reinvent the wheel, especially under deadlines. If my end goal is to bring the city alive, whether I achieve it quickly and realistically using video footage of traffic or spend weeks hand-crafting each street to detail in 3D, it doesn’t matter. Now of course if your goal is to reinvent the wheel then yes, by all means, take the time and effort to handcraft every detail, it would for sure be more rewarding in the end albeit taking a significant amount of time.

And since time wasn’t the luxury I had, the first thing I did was go to Pexels. Pexels is an amazing community of creative individuals who are generous enough to donate their high-quality photos and videos for free for any use, commercial or non-commercial. I highly recommend this website for all your stock footage needs. I ended up finding a lot of drone footage, shot at night however most of them were looking at a three-quarters angle rather than perfectly down. This is important as when you assign this video footage as texture to your ground in UE, the texture needs to be as flat as possible and devoid of any camera moves. In the end, I found these two videos for which I am thankful to Tom Fisk and Mixkit as they were perfectly straight down looking drone shots.

As the camera is moving and rotating, I ended up first stabilizing the footage in Nuke (you can do it in After Effects as well) so the roads were locked in place. Then I used UE’s Media Player components to load these videos in.

To set it up, here’s the step by step breakdown:

1. Right-click anywhere in the content browser and create Media > File Media Source. Double click it, this is where you enter the file path to your downloaded videos.
2. Next, we need a player (imagine like a VLC media player) that will play this video and a video texture (like regular textures that we can call in Material Editor). To do this right-click again and create Media > Media Player, it will ask you for convenience, if you would like to create the output from this player to a video texture and link it up. Check the box in that pop-up and click OK.
3. Now double click the Media Player and you should see a window with play/pause buttons and in the list below, you will see your Media Source from step 1. Double click your Media Source to start playing it. Depending on your hardware, the playback would either be smooth or choppy. Make sure to click the Loop checkbox to have your videos loop. In my case, since each of these traffic videos was pretty long, I never ended up seeing the end of the clip in my camera animations. If your camera motion is long and your playing video is shorter, you would see it jump back to the start so make sure your video is loopable before you import it.
4. Now, let it play here and minimize the window. Do not pause your video or close this player window. As in the next step, we will create the material and assign the video and in order to see the video playing in the editor, you have to keep the player window open. This one little trick took me days of nonstop searching to figure out. Without seeing the videos play in the editor, there was no way for me to block my city. However, if you do close the player, this will simply pause the video textures on your ground, which is fine if you are running low on resources.
5. Right-click anywhere in the content browser and create a new material. Set the mode to Unlit and drag and drop your Video texture that got created in step 2. If you followed step 4 correctly, you should see the little thumbnail playing. Hook it up to the emission color and save your material. Now if you assign this material to a massive plane and put it on the ground level, you have an animated video!

Since our ground was already covered with those repeating window textures that we talked of above, these road videos actually started covering that. To make the road videos transparent (so the black areas will become transparent and the roads stay opaque), I simply changed the Material mode to Masked/Translucent. Then I used the Cheap Contrast node to turn the video into a black and white map (white is where light is and will be opaque, black is the dark areas around the roads which will become transparent) and plug this into the Opacity slot. Now I can see the roads while also seeing the repeating ‘window textures’ ground material.

Here you can see the base ground material which is the ‘window textures’ just repeated a lot.

The 3D buildings come on top and clearly hide a lot of those repetitions. However, the city still looks uninteresting.

The roads definitely bring the whole thing together by adding tons of surface details.

Lastly, something important to realize is that despite adding these traffic videos, they still feel a bit separated from the city. To immerse the city in utmost realism, we need to have these lights from the streets and from the cars, actually, light up our 3D buildings.

This is what I did in the final shot where people were rioting against the police. I found this perfect video on Pexels uploaded by Kelly Lacy, stabilized it and brought it in, and set it up the same way as before. 

However, you can see the video already has some buildings in it that are skewed and distorted so I placed my 3D buildings exactly on top of them to hide that. Now my buildings were so close that the flashing lights from the cop car should technically light them up. So I created a Rectangle Light and assigned my luminance keyed riot video as described above, into the light texture. Aligning it perfectly meant that whenever the police light flashed bright, my buildings would automatically be lit by it. This absolutely sells the realism and removes all doubt from the viewer's minds that the buildings are not real.

3D buildings on top of riot video (No interactive light)

3D buildings are now illuminated by the riot video by using a Rectangle Light with video texture.

Camera Animations

Your last percentage of adding realism to CGI is through camera animations. As the city started to take shape, I was tempted like a kid in a candy store wanting to fly around and put cameras in so many exciting spots and animate them in very exciting ways. However, it was very important to realize that impossible camera moves (camera flying way too fast, rapidly flying way too close to objects, flying through things like fences or glass, flying too abruptly and changing directions quickly, etc) easily take the viewer out of the immersive experience.

So more than anything, always do realistic camera moves. The way I like to think about it, real-life cameras are very expensive and therefore filmmakers don’t do these crazy camera moves. Just like that, imagine that your CG camera is actually an expensive piece of hardware that will break if you are not careful. This will let you focus on moves that are humanly possible resulting in a much more convincing animation.

For this entire short, I chose to have camera moves that could be recorded by a helicopter/drone, and hence my movements were very slow. I also made sure none of my camera angles were too shallow (looking at glancing angles) as this showcases how flat my road video textures are. Always work smart and play to your strengths. If a camera view is showing some issue, it is better to find another angle rather than break the viewer’s immersion by showing something incorrectly.

Having said that, I really wanted to immerse the viewer completely in this world that I was building, which is why I chose to add a handheld camera move. I flew around the city for hours trying to find a spot, a perfect rooftop view of the whole city where you could detach yourself and see the world as it passes by.

The shot that shows up at the 2-minute 11-second mark is set in the rafters on the side of a building which is all hand-animated in the base pass. I wanted it to feel like a DSLR camera that one of those rooftop climbing daredevils have, with autofocus on. The look of bokeh for the city lights definitely was appealing and autofocus usually ends up doing it naturally.

At one point as he is trying to navigate around the rafters, he suddenly hears a loud distant bang, pulls up the camera to see something but is unable to find it, and ends up searching by panning the camera around. This was the base pass that I hand-animated. To add realistic camera sway I decided to use the Camera Shake blueprint that comes packaged with UE.

The complete guide on how to create it is available here on UE’s YouTube channel:

One last thing to adjust in your cameras (or you can do this in a Post Process Volume) is Exposure and Raytracing settings.

By default, cameras are set to Auto Expose which means if it is dark, it will automatically brighten your scene or vice-versa. I personally don’t like it and instead set it to Manual. After that adjust the Exposure Compensation slider to your taste to make it bright enough to be a nighttime scene but not too bright. In real life, if you fly over cities, the dark areas are pretty black at night time.

Ray-Tracing

Lastly, I wanted to leverage the power of my Nvidia 2080Ti GPU which was RTX compatible so in Camera > Lens > Rendering Features  > Reflections I set to Ray-Tracing.

NOTE: For Raytracing to work make sure to go to your project and search the setting called “Default RHI” and set that to DirectX 12. Also, search Raytracing and make sure it is turned on.

Next in the Ray-Tracing Reflections section of your camera (or post-process volume), I set the 
“Max Roughness” to 1. When it is set to the default value of 0.6, what it means in layman’s terms is that UE will try to compute reflections using Ray-Tracing until a Roughness value of 0.6 and the rest will be computed by using old school “Screen Space Reflections (SSR)” technique. When you set it to 1, this will slow your GPU down but in turn, your entire scene will start using Ray Tracing, thereby eliminating the less accurate SSR. If your scene is full of rough surfaces, adjusting this won’t matter much.

After that, I set the Max Bounces to 16. Default is 1 and what that means is a light ray will bounce off 1 surface and then die (not compute). If you set it to 16, the light will bounce 16 different times on different surfaces before dying. This means you will get great internal reflections of objects reflecting in each other. This will also slow your GPU down.

Next is Samples per pixel. Increasing this value is what slows your GPU the most. How high you set this depends purely on your GPU, mine could handle 16 samples which in turn slowed the scene from 45 FPS to 10 FPS. However the higher you go, the cleaner your reflections will be with less noise. And for rendering cinematics, fps doesn’t really matter.

I also checked the “Include Translucency” checkbox as this allows Translucent materials to show up in reflections as well. And that’s pretty much it! So for a final recap, here’s another camera angle at different steps:

Buildings scattered and materials assigned.

Neon signs and animating holograms added.

Physical lights added to the scene to light the buildings up.

The Exponential Fog added, camera exposure adjusted resulting in final render for compositing.

Sound Design

Any great immersive experience cannot be complete without sound design. Good video is half the battle, as they say, good sound is the other half. I sound designed the whole short using exceptionally stunning tracks already created and given for free by the talented musicians without whom this short wouldn’t have evoked any emotion:

Eon: Eon’s royalty-free soundtracks collection is massive and his track “Fragments” had a very intriguing opening that allowed me to create this opening title shot. The opening sunset shot and the following city establishing shot are actually two different shots that are merged together in After Effects using 2D cheats to make it look like you were traveling down an elevator shaft.

Scott Buckley: His soundtrack “Discovery” is the most prominent theme throughout the short. The whole track flows perfectly as is, however, I wanted to key at the moment when the track drops the pace and becomes mellow. I edited the short and the soundtrack in a way that when that happens, it cuts to the handheld shot high up in the rafters which is a very personal moment.

I further wanted to immerse the viewer at this moment and make them feel what it's like to be viewing this futuristic city from far away. This is why I got sounds from Freesound which is my go-to resource for diegetic sounds. These are sounds that are created within your narrative. In my case I looked for sounds that sounded like metal clanking, old rusty metal doors swinging, metal chains hitting each other, distant city rumbles, distant car sounds, and just strong winds blowing high up on skyscrapers. Mixing these in the right amount and the shot elevates to the next level.

Having people comment later that that was their favorite shot, definitely makes it worth the effort that went into designing the sound for it.

Conclusion

In the end, I absolutely challenged myself creatively in this short span of 4 weeks and will definitely recommend people to enter contests where deadlines are a real thing. Having that pressure definitely focuses your attention and gets the gears rolling faster and smarter. I am genuinely thankful to every single person who helped me out throughout this journey. I am hopeful that the information in this tutorial will help some of you out.

I am definitely going to take all the things I learned in this into my next project where I tackle to recreate the intro of the 90’s Batman: The Animated Series with movie-quality CGI. I am planning on taking the classic 40’s Art Deco style from the show and marrying it with the Cyberpunk aesthetics in a neo-noir palette. So if you would like to follow the updates, check out my socials where I would be posting the journey and take you through a re-envisioned Gotham and Batman.

Thank you for your time, I bid you farewell, and good luck in creating your own worlds!

Indrajeet Sisodiya, Senior Compositor/Environment Artist

Interview conducted by Arti Sergeev