Using Photogrammetry to Make Plant Pots with ZBrush & RealityCapture

Introduction

Hey, I’m Muhammad Awais Azhar. I’m 28 years old and live in Pakistan. I’m a self-taught 3D Artist. I got a Bachelor's degree in Information Technology (IT) from GC University Faisalabad in 2016. I started 3D as a hobby around 2014 but after graduation, I started to work as a freelance artist. I mostly work on props, product visualization and environment art. I also really love to explore the power of Unreal Engine.

Photogrammetry

I started to explore photogrammetry, or 3D scanning, back in November 2021. At first, I used the most popular free software called Meshroom. It is a good free option but it was a bit slow for me, so I started to explore more options for photogrammetry.

RealityCapture is a very easy and intuitive software. And the thing which I loved the most about it was its speed. It is a GPU-based software, so it is very fast in processing images for alignment and mesh reconstruction.

The Old Plant Pots Project

For photogrammetry, old, rusty, dusty, and matte surfaces are ideal for scanning. So I had a few old, a bit broken and textured pots available at home that were ideal for this purpose. And I also wanted to use them in my projects.

I used a mobile phone camera to capture the initial images. Usually, we capture pictures from all possible angles: front, back, top, and down. For this specific project, I didn’t capture the bottom side of the pot. you should loop around the subject.

Capturing Data

There are a few things you should consider before taking the pictures of a subject. 

• Lighting conditions: there should be no harsh lighting on the subject that creates shadows. The subject should be evenly lit, which means there should be no shadows on the subject, otherwise, they will be baked in the Texture Map and we won't be able to light the subject properly in 3D (because there would be shadows already in the map)
• For even lighting, if you are shooting outdoors, then I suggest overcast weather or at least shooting in a shady area. Overcast weather works as a softbox and creates even lighting.
• Pictures should be sharp and clear, there should be no blur while taking pictures. For that, you can set the high/fast shutter speed (if you have manual control on the camera).
• Enough overlapping: while taking pictures, you should keep in mind that there should be decent overlap between images, I mean, if you take one picture, then the other picture should have at least 30-40% of overlap with the previous one, that will help the software in the alignment process.
• The subject should be still, don’t move the subject while taking pictures, otherwise, the software will not align the cameras.
• Take as many pictures as you can, you should not limit yourself while taking pictures. The more pictures, the better the quality.

RealityCapture

Before putting the images in RealityCapture for the process, I used Adobe Lightroom to process the photos. I did the following:

• Decrease the contrast
• Remove the shadows by decreasing the blacks from the picture
• Remove the highlights 
• Sharpen it a bit 

First, I imported the processed images into RealityCapture by clicking on the Input button. Then I clicked “Align Images”. On the left side, as you can see, 73/73 cameras are aligned because of a good overlap and created mesh points.

Mesh Construction

First, we have to adjust the construction region so RealityCapture doesn’t have to create and process unnecessary information/data (by clicking on these colored points). After this, you have to select the mesh quality from the Reconstruction tab. I selected the Normal Quality, which is usually enough for this type of scan. It took around 6 minutes to complete, which is very fast if we compare it with other software.

From the Scene tab, I picked the Rectangle tool to clean the mesh in RealityCapture. By pressing the CTRL button, you can add to the current selection. Then I clicked on the Filter Selection button to remove the unnecessary mesh. I exported the mesh in the .obj format to import in ZBrush for further processing.

Before decimating the mesh, I deleted the bottom base and then used DynaMesh to close the hole and to even out the topology. Then, from the Zplugin panel, I pre-processed the mesh and then decimated it (twice). After decimation, I further reduced the polycount by using ZRemesher. It also turns the mesh into Quads. So before decimation, the total polycount was almost 2 million, and after decimation and ZRemesher, the low poly version had almost 5k polygons. 

After creating the low poly version, I used the Project option to project the details of the high poly to the low poly.

I just simply went to the Zplugin panel again and clicked on UVMaster. I used the lowest poly version (5k), then clicked on the Unwrap button, I'm still improving the UVs, but for this tutorial, I did that. Then, I exported the medium polycount version to RealityCapture to project the texture on our newly UVed version. The important thing which you should consider before exporting is that it must be exported or replaced on the same mesh (.obj) file that was exported from RealityCapture before (after the cleanup). If we don't do this, our newly created model will not be in the right place for texturing.

Texture Re-Projection

I imported the model in RealityCapture, it replaced the existing mesh. Then I selected the 8k texture and clicked on the Texture button. After creating the texture, I exported it.

I imported the low and high poly versions in Marmoset. Then I clicked on Bake Project to bake the details from high to low poly mesh in a Normal Map. I used the albedo/diffuse texture, which I created in RealityCapture. 

Rendering

I created a simple scene in Blender to render the models. I used one rectangle light in front. HDRI from Poly Haven was used as the main lighting source. The scene composition was heavily inspired by Quixel renders.

I created this turntable animation in Marmoset Toolbag. The subject is only lit by one HDRI, which is placed in the Sky option.

I followed the same process for the other two plant pots. 

Conclusion

It was a great pleasure and honor for me to share my workflow with this amazing community. If you still have any questions, don’t hesitate to ask me. You can contact me on my ArtStation or LinkedIn. Once again, thank you very much for your time.

Muhammad Awais Azhar, 3D Artist

Interview conducted by Arti Burton