Recreating the Athena Statue Using Photogrammetry, ZBrush & Marmoset

Photogrammetry is getting more and more popular, allowing the creation of extremely realistic renders by using photos of real objects. Maxime Marier, a teacher at Isart Digital, Creative School in Video Games & 3D-VFX, has shared the work process behind creating the Athena statue using photogrammetry.

Introduction

One technology I always wanted to try since it came out was photogrammetry, the process to create a 3D model of a real-world object with high-quality pictures. From what I’ve seen, no other techniques achieve the level of realism that this technology can.

The Athena Project

One project that made me want to try this technology more than any other is Star Wars: Battlefront. In the new EA version from Dice Studios, they used photogrammetry to make the levels of the games more believable than ever. As you can see in the picture below, they did go to the set of the original Star Wars trilogy to take high-quality pictures of the set. With those pictures, the team at Dice was able to recreate each asset of the set with an incredible amount of detail. It was more than enough to spark interest in that technology in me as a Star Wars fan.

I highly suggest you watch the Dice GDC talk about the photogrammetry process used in Battlefront, it is available on YouTube. Having played the game, I have to say that it does feel like you are right there on the set of the original trilogy.

This GDC conference from Dice was a great starting point for me. The first thing I understood is that I needed some high-quality photography. I do have some basic knowledge myself, but I am no photographer. At the time, I was working at Institut Grasset in the 3D Animation department. One of the subjects taught there is TV production and Movie Making, and, as you probably know, mastering a camera is a fundamental part of those classes.

So I decided to ask my colleague Jonathan from that department if he wanted to help me to take high-quality pictures of a monument right here in Montreal. Fortunately for me, Jonathan was interested to help me for a few hours of the photoshoot.

The Photoshoot

The first step of the project was to choose a subject, in that case, a monument or a statue. This website about public art in Montreal was helpful for that task. When browsing the website, I came across a statue of Athena located on Jean-Talon. At first sight, I knew it would be an interesting subject for our photogrammetry project.

So we took a car, a big ladder and professional camera equipment and took off to Athena’s park. Right away I was happy with my choice because it is indeed a beautiful piece of art. It was quite a challenge because the height of the statue is 2.31 meters and getting the full coverage for the pictures was difficult.

The process of shooting a subject for photogrammetry involves taking pictures all around the object to cover 360 degrees from multiple angles. As you can see in the picture below, we took a full 360 of the statue from 3 angles for maximum coverage, we also took some close-up shots for extra details. The blue squares in the image below represent every picture taken during the shoot, the software is able to calculate the position of every picture from the area when you input each individual picture.

That makes for a total of 336 pictures and nearly 10GB of raw pictures. Jonathan provided me with compressed JPG and RAW files, for this project I choose RAW for maximum control of the images.

One thing that was difficult to control in this shoot was the lighting conditions. The shoot itself can take some time, so the lighting conditions will change during the day. The day we did the shoot, it was partly cloudy, and you can see the cloud pattern moving on the statue itself, which is not ideal. I think to have the best results, you need to choose a cloudy day, so the lighting is as diffuse as possible on your object and you don’t have shadows that can impact the result of the photogrammetry. That is for an exterior shoot, of course, if you can bring your subject inside for a real studio shoot, it's the best thing you can do.

You can see the settings I used for the processing of the raw files in Photoshop, the camera settings are also there. The point of the settings used here is to get the maximum information from the raw files and avoid dark shadows or burned highlights that can lead to a loss of information in the pictures. I exported the final result as uncompressed .tiff files.

Generating and Processing the 3D Model

I ended up testing two softwares for processing the pictures and generating the 3D model: Agisoft Metashape and Reality Capture. The result I had was that the model from Reality Capture seems slightly better than the one from Metashape, but they were really close.

The main problem I had with the generation of the high-resolution 3D model is that the model generated from the full statue pictures did not have enough details. One simple explanation for that is that when you have the full statue in a picture, there is a loss of space around the statue, too much to be able to fully capture the subject.

What I ended up doing is generating a model from the close-up shot of the head. Once I had the full-size model and the head, I was able to stitch both those models with the ZBrush's DynaMesh. In ZBrush, I also did the final step of cleaning the sculpt with the sculpting tools. As you can see in the final result, I was able to keep the texture data safe by transferring the texture to ZBrush vertex color data.

Optimization

Now, with the high-resolution model scan of my statue in hand, I know I needed to do some optimization to have a model that works well in a real-time rendering engine. There are some tools for automatic retopology, but I wanted a level of detail those tools could not do, so in the end, I chose to manually make a new topology with Maya's Quad Draw tool. Doing it manually does take a considerate amount of time, 5–6 hours in that case, but I feel the level of detail of the final render was worth it, I also wanted to do justice to the original work.

Materials in Substance 3D Painter

Now, with the low-resolution model done, I was able to bake a good Normal map in Substance 3D Painter with the high-resolution scan data, I also baked the color data from the scan. For the final material, blending the color data with a copper material for the statue and a granite material for the base, I was able to fine-tune the mix between the two.

Final Renders

For the final render, I had to choose a tool. I wanted to render it in real-time because I mainly do work related to video games. I do love my Unreal Engine, but it can become time-consuming to get renders out of it, so I decided to go with Marmoset Toolbag, which can get you some professional-looking renders in a few minutes. So here’s the final render of this photogrammetry project.

Conclusion

In conclusion, I can say that some things I did could be improved in a future photogrammetry project. Choosing the right day with the right lighting conditions can make a great difference. Also, one thing that I've learned is that close-up shots do really work well and stitching the model in ZBrush also works with the DynaMesh tool, so I would probably make more close-up shots.

I am happy with the final result, I think at the end of the day, we accomplished our goal of scanning the statue of Athena and rendering it in a real-time render engine. Huge thanks to Jonathan Bélisle for his help with the photoshoot, I could not have done it without his photography skills.

Maxime Marier, Teacher at Isart Digital