Making a Destroyed Car Model with Photogrammetry

Introduction

Hi, my name is Kyle, I’m a 3D environment artist currently based in Wellington, New Zealand. I studied at AIE (Academy of Interactive Entertainment) in Canberra, Australia, and graduated back in 2013.

After a few short 3D contract jobs I got a contract to work for Eye Candy Animation in Canberra for close to 2 years as a 3D generalist working on a range of projects from tv advertisements, educational mobile games, AR (Augmented Reality) apps, and interactive museum exhibitions.

In late 2017 I was hired by A44 (Aurora44 at the time) as a 3D environment artist to work on their project “Ashen”. I worked there for about 3 years and jumped between environment art and level design to help bridge the gap between art and design.

I also worked on Ashen’s Nightstorm DLC primarily as a level designer and assisted in kit blockout, level dressing, and encounter setups.

Like a lot of us, I loved playing games as a kid but as I got into my teen years I started to really want to “make” games. I would create multiplayer maps using Far Cry map editors for my friends and me to play together and eventually started to gravitate towards the 3D art side of things.

I started getting interested in photogrammetry when I came across Alex Alvarez’s “Creating Natural 3D Environments” talk at Gnomon (see below). I never really actioned anything for quite some time but I did do the occasional photo shoot of objects or surfaces whenever I was out during an overcast day in the hopes that I would eventually process them and use them as a model.

Photogrammetry Car: Taking Photos

I was running some errands one day and drove past this burnt-out car that was abandoned in a storm water canal. I decided to pull over and take some photos with the intention that I could use it for a photogrammetry project. As I didn’t plan for this and just came by it, unfortunately, all I had on me was my phone (Samsung Galaxy S6). If I had more time or knew where the car was beforehand I would have at least brought a proper camera.

It just seemed like a good opportunity at the time as it was an overcast day, the car was burnt out so there weren't many areas with reflections and since it was left in a storm water canal it would probably be removed in the next few days so I felt I had to take photos there and then. Weather conditions were perfect for taking photos for the purpose of photogrammetry as the shadows would be much less of a problem when working on the albedo.

Unfortunately, I was pretty short on time when I was taking the photos so I only took 86 images of the car, which is not nearly enough for something of this size and complexity.  If I were to do something of this scale again I would be aiming for roughly over 300 to make sure I captured as much as I could.

When taking photos of the car I took a few steps to the side each time, eventually rotating around the car, and as I made a complete loop I raised and lowered the camera so I could capture other areas of the exterior and interior of the car.

Photogrammetry in RealityCapture

RealityCapture is super streamlined and pretty straightforward to use. Tere are a lot of resources out there to help learn the program but I found Michael Pavlovich’s photogrammetry videos to be a huge help.

I placed all my photos into a folder, then opened RealityCapture and clicked on the “Folder” option under “1.Add imagery”.

Once all the photos are loaded into RealityCapture, go under the Alignment tab and click “Align Images”, this will build the point cloud data for the mesh to be generated from.  Sometimes this doesn't always work out and you’ll need to adjust some settings or place some control points on your images to help RealityCapture build a more accurate point cloud.

I followed this tutorial from the RealityCapture youtube channel for placing control points on your images:

Now that I had the point cloud built I adjusted the reconstruction region to fit around the borders of the car to ensure that when I build the mesh it will only be generated within the volume.

To make life a little easier as well I set the “Ground Plane by Reconstruction Region”, this will just reorient the project to align the world grid to the reconstruction region.

From here, I go to the “Reconstruction” tab and click on “Normal Detail” – this will give you the high-resolution mesh. From here you can start to get an idea of how much clean up it will need, what areas need rework either in ZBrush or adjustments in RealityCapture settings.

I then click on the “Texture” button to add the albedo to the model, then hit the mesh button under export and you will have a high-resolution model with a 0-1 albedo file.

Model Treatment

The biggest challenge with this project was patching areas that were not captured in the initial scan.

I cleaned up the high-resolution model by importing it into ZBrush to remove noise in specific areas and to patch the major holes in the geometry that didn't get created in RealityCapture.

I particularly focused on the roof as that was the most visible and broken area of the high resolution. I did this by appending in cubes, scaling them to fit the hole, merging it with the base model, and blending it in by sculpting and stamping details into the model.

Once the model was cleaned up as best I could for the time, I created a low poly version of the car in Autodesk Maya, unwrapped it, and baked the high res mesh to the low poly in Substance Painter.

Once it was all baked I made a pass over the model using the clone stamp tool in Substance Painter to patch the areas that were baked incorrectly or didn’t have the correct albedo colors.

For the vegetation and rubble, I just used assets from the Megascans library and Quixel’s Bridge to UE4 tool to easily get the assets in engine.

Texturing

Since most of the model had the Albedo and Normals from the bake, I mostly needed to focus on patching spots and adding roughness and metallic details to it. However, painting in different materials across the entire model would be too time-consuming.

I ended up using Photoshop’s “Replace Color” adjustment to create masks that I could apply in Substance Painter to mask out different materials that I grabbed from the Substance Source library.

Rendering

I wanted to set up the scene so the viewers could focus purely on the asset itself without being distracted by the environment around it, however, I did notice that adding in some foliage and rubble around the vehicle helped ground it a little more.

I set up the scene with one directional light, sky atmosphere, sky light, and a reflection capture and decided to just go with RTX lighting as I couldn’t really be bothered to bake and set up lightmaps. I then added the cameras to the scene so I knew what angles I needed to focus on the most.

Then I added a bunch of dynamic rectangle lights to help brighten some of the darker areas of the car and also make the model pop from certain camera angles by following a three-point lighting method.

After I was happy with the lighting I started playing around with the post-processing volume. I just added a little bit of vignette and increased the contrast a little.

Is It Ready for 3D Printing?

I created the model with game specs in mind. I’m currently using two texture sheets and it’s sitting a little over the 36k tris at LOD 0.

If you were to try to 3D print this though, it would need a lot of work done in ZBrush to clean up the mesh by removing any floating geometry and thickening up areas that might get too thin and snap off during the printing process.

Getting Started with Photogrammetry

Photogrammetry is really affordable, CaptureReality is free to download, however, if you want to export your mesh it will charge you depending on the amount of detail that model has, and since I used my phone and a minimal amount of images, getting the car processed was really cheap.

If this project taught me anything, it is the fact that you don’t need much to start. If you’re interested, have a phone and a decent computer, you can go out on an overcast day and start shooting.

The biggest things I would emphasize with photogrammetry are:

• Go out on an overcast day (shadows are your worst enemy)
• Make sure your camera’s settings are locked before you start shooting (you don't want to adjust anything with the shutter speed, exposure value (EV), or the ISO values)
• Take lots of photos!!! (try to capture as much as you can but remember to take overlapping images and avoid rotating the camera)

Below, I have linked some resources I found the most useful when looking for information on photogrammetry:

• RealityCapture Youtube Channel
• Michael Pavlovich's Youtube Channel
• Grzegorz Baran’s Youtube Channel

Kyle Chin, 3D Environment Artist

Interview conducted by Arti Sergeev