Making a Realistic-Looking Portrait in ZBrush & Marmoset Toolbag

Introduction

Hi! My name is Milton Fernandes, I’m a 3D Artist based in Lisbon, Portugal. I’ve worked mainly as a generalist for a couple of years but recently started to specialize more as a Character Artist.

I was introduced to 3D thanks to Odd School, a digital arts school here in Lisbon, where I first came into contact with 3D modeling and digital sculpting. After that first contact, I never stopped! I was immediately hooked on learning the craft and was shortly employed in a production house where I worked mainly in motion graphics for television and web for a couple of years. That was a great first start for my career, we had a lot of work to do, and had to come up with creative intros, trailers, and animations for a variety of broadcasting channels and clients. It was the perfect place to start learning the pipeline of digital art production and develop my skills as a professional and artist.

A couple of years later, I got the chance to join Studio Nuts, a digital production studio mainly focused on delivering high-end 3D and 2D illustrations and animations for the advertisement industry. I got the chance to work for very big clients like BBDO, Havas Worldwide, and Ogilvy, and (more importantly) really hone my skills and quality output.

Right now, I’m focusing my skills mainly as a Character Artist, working mainly as a freelancer. 

Starting a Portrait

I’ve been wanting to try out a new workflow for character production for real-time applications with a focus on likeness. I’m greatly inspired by artists like Ian Spriggs, Kris Costa, Vimal Karketta, Şefki Ibrahim, and Marlon R. Nunez.

I became aware of Art Heroes’ Likeness Course after a few web searches, and since I’ve been a big fan of Marlon R. Nunez's work for a long time, I immediately seized the chance to join the first class!

For my subject, I decided on a digital double of Lupita Nyong’o, specifically a particular photoshoot of her that I find very visually striking. My goal was twofold: to learn with the ArtHeroes team, specifically the workflow aimed for real-time production, and to have a nice portrait of Lupita in the style of these reference photos.

As with every project, it all starts with a thorough search for a good set of references, Lupita and anatomy. Pinterest and Google are the way to go (at least for myself) in this regard, but I also have some books that helped me along the way: Anatomy for the Artist by Sarah Simblet, and Anatomy of Facial Expressions by Uldis Zarins to name just a few.

I used PureRef for collecting and organizing everything. There’s this very tricky thing when it comes to likeness because you need a lot of references from many sides and angles, but you have to be extremely careful because they can harm you – you need to take into account the distortion of camera lenses, how the light conditions “paint” the face, the age of the subject when the photo was taken… If you are not careful, you may be misled often and work around in circles. For that, I found it to be very, very useful to choose one to two photos at max as my main reference – if I ever felt lost or overwhelmed, I could easily come back to those and remind myself of what I’m aiming towards.

Working in ZBrush

After gathering all the references, I started from a good old sphere in ZBrush and started sketching my base head: the idea here was to create a very low-resolution sketch of a female head, focusing on important anatomical landmarks to get a good base to start with. I was not focusing on likeness at this point, just on having a good semi-generic face. 

After I had a base I was happy with, I tweaked and sculpted it to match it with my references. This is where the actual likeness phase begins and is also where good reference material and good observation are paramount. It’s a balancing act of finding how the main shapes of the face give the personality and uniqueness of the face. For some references where I could somewhat match the camera angle, I used Spotlight to trace some elements. 

This phase took me some time until I felt I was getting somewhere. It was not uncommon to close a sculpting session one day, happy with the result, only to come back the next day and hardly recognize what I was happy about the day before! 

The key principles here are observation and patience. It’s all about really observing and comparing what you have in front of you, and your references. Do not try to sculpt anything from memory or imagination, as those are unreliable. 

I focused mainly on primary forms at the start. Secondary and tertiary would come later only when I felt that the likeness was somewhat achieved with only the primary forms.

When I felt like I was in a good place to add good topology, I wrapped a base mesh around the sketch to have a proper topology to work with, and now it all came down to refining everything even further. For the wrapping process, I used ZWrap, a very powerful plugin for ZBrush. I used a previous BaseMesh I worked on in past projects, packed with UV’s and a set of eyes and teeth (they were done for a previous likeness attempt I did).

Now with good topology, I kept refining the face further to get closer and closer to the likeness. There are no tricks I can really share here, it all came down to focused hard work during my daily sessions, evaluating what was working, checking different angles, and seeing how everything’s working. It’s not easy, and I felt stuck many times. In the end, as Marlon said a couple of times: “you have to trust the process”. Not rushing anything, slowly I started to see some good progress on her.

When it came to the skin detail, I used a pack of multichannel face skin maps from TexuringXYZ to project the fine porosity displacement details as well as a Diffuse Map. I again used ZWrap for the projection. These maps only cover part of the entire face though: here I create more sculpting layers to refine the detail, especially in sensitive areas like eyelids, ears, and in tight spots where the projection didn’t do a good job or where there was no detail to project from, like the neck and entire back of her head. On some of the tricky areas, I had to entirely ditch the information baked from the maps and re-sculpt with the help of some alphas and custom-made brushes, like her eyelids.

For the rest of the face and neck I used a pack of skin alphas from Texturing XYZ, custom brushes I created in previous projects, and also a female scan from the 3D Scan Store for reference. I find scans to be incredible resources to study inside ZBrush not only how the porosity changes throughout the entire head, face, and neck, but also the intensity values. 

I kept all the fine detail stored inside sculpting layers. This way I could always go back and scale down (or up) the intensity values of some porosity areas. I like to work as non-destructively as possible, although this means that I have to be organized at all times. 

Even at this stage, always referring back to my references was important, as the skin also adds a layer of realism and likeness that could make or break the model. 

It was at this point while studying anatomy to push my likeness further and further, that I thought it would be cool to up the difficult challenge and create a smiling expression on top of the neutral one.

This would help me deepen my understanding of FACS and result in some extra portrait renders. A great resource I used extensively during this phase was Melinda Özel’s Instagram page! Highly recommend everyone to check this page out for great FACS research and references.

Again, patience was key throughout the whole process. Instead of large strokes and bold changes, I worked deliberately slow, small tweaks here and there, always paying attention to how the topology was working, respecting the natural folds and creases that form with the muscle movements that give that characteristic expression. I don't use any special brushes or tricks inside ZBrush. Usually, I play around with the brush modifiers to get the desired effects that I want when sculpting. 

From left to right you can see the progression of the smile, and how very small tweaks change how the expression is reading.

Finally, I added a very simple pose, and tidied everything together, with layers for each morph expression.

The hair was done inside ZBrush with the FiberMesh system. Being a very low-cut hair meant I didn’t have to do a lot of manual brushing, but I did spend a couple of hours experimenting with the different settings to get the result I wanted. 

An early test version of her expression and hair:

Making the Outfit

I’ve worked with Marvelous Designer professionally in the past, so this part was very straightforward without a lot of guesswork. Since I was planning on a more “posed” look for the final portrait, I used a generic female BaseMesh and roughly tweaked it to simulate the pose I wanted the garment to simulate. 

I simulated a very basic garment, based on the clothes she’s wearing in my main reference photo. 

I then exported the cloth mesh with a very high destiny back to Zbrush and used ZRemesher for a quick retopo. I took care of maintaining polygroups for each cloth panel during the entire process so I could then use the Panel Loops feature to give the entire clothing some thickness.

Retopology and Unwrapping

Having a previous head mesh with UV’s was a time saver in this aspect. It meant I didn’t have to spend time retopologizing everything from scratch. This was also a time saver when it comes to unwrapping, with the bulk of the work being on making sure I had a very good wrap of the base mesh to my sculpted-based head with the topology flowing nicely.

For the clothing, the UV’s were done using ZBrush’s UV Master plugin: I unwrapped by polygroups to have neat UV islands to work with.

Texturing

As mentioned previously, I used an XYZTextures Albedo Map that came with a pack I purchased as a starting point. With the help of ZWrapper, I projected the skin color information to my mesh and painted the rest of the head with Polypaint. 

This got me a great starting point, but still far away from the type of skin I intended. I used a combination of Polypaint and Photoshop to clean the base texture: first by cleaning contact shadows from skin folds and trouble areas like the ears. The maps also come with hair, eyebrows, and eyelashes baked in, so I needed to completely erase those as well to end up with a clean map. 

After that was done, I used a combination of spray brushes with skin alphas in ZBrush to get the right variation of skin tones and detail I needed. 

Left to Right: Base map, Repainted Texture

I also used the Adjust Colors Option on ZBrush to tweak things like contrast, hue, saturation, and intensity values. These sliders can be used in conjunction with the different masking options. Finally, I exported the new Diffuse Map from ZBrush and other additional Maps like AO and Concavity Maps. I blended these in Photoshop and tweaked them to create other supporting Maps that I would need in Marmoset.

For the clothing I went with a very basic approach: texture-wise I used a blend of two TexturingXYZ Micro Fabric Maps just enough to break up the surface of the cloth in the final render. 

For both the clothing and earrings I took advantage of Marmoset's great shader library to have a great starting point for the shaders.

Rendering

I used Marmoset Toolbag 4 for the final scene render and took advantage of the new Ray Tracing feature.

For the head model, first I exported all the expressions out from ZBrush and into 3ds Max to create morph targets and an animated mesh cycling each expression.

With the final animated mesh ready, I exported it as an FBX, which I then imported to Marmoset and keyed each expression frame to an animation takes. This way I could easily choose which expression is visible on the viewport, or choose to simply have them cycling through.

For the lighting, I was aiming for a very contrasted “studio lighting” feel, so I created basic three-point lighting with a backlight, the main spotlight, and a very dimmed skylight to help bring some areas out from the very dark shadows. 

The skin was by far the most challenging aspect of the scene setup, as well as where most of my attention went to. The process was basically trial-and-error until I was satisfied.

The Specular and Gloss maps were hand-made in Photoshop, using all the maps I exported from ZBrush as a base and tweaking them. Having a good knowledge of how these maps work will help you in experimenting a lot and not being tied down to specific software or technique to achieve the results you want. For some things, I had to revert back to ZBrush and paint some custom maps to add to the composites inside Photoshop. It was a messy workflow, but it was also freeing in the sense that I got to explore a lot. Not to mention, I got to see Marlon’s technique during the course and use that as an inspiration for my approach.

The same workflow was used for the Transmission maps, necessary to have a nice SSS working on the skin.

For the Surface Detail maps, besides the Normal Map with all the baked details I sculpted inside ZBrush, I also added a tileable micro Normal Map to further break the skin surface. 

Conclusion

I didn’t keep track of the time I took to develop this project. It was done always during nights and weekends since I still had my studio day job and other personal projects rolling. 

The Likeness program kicked off right at the beginning of February, and we had our final deliveries scheduled for mid-April, so all in all we had plenty of time to go over all the classes and resources that were available (and there were a ton). This also meant I had the time and space to experiment, test, and learn new techniques.

During the program, we had bi-weekly review sessions and a community group where we always had plenty of chances to share and comment on everyone's progress. Having a community like this, and getting inspired by everyone’s progress is super helpful, and it’s also a lot more fun!

Throughout the project, my main challenge was more mental than technical: sometimes it’s easy to feel like the project is not evolving as you wished, and that your skills are limiting the progress. Most of the time, it was always a question of taking a couple of steps back, rest, and come back with fresh eyes. The likeness is really tricky, but that’s also why I love it so much! There’s so much room to experiment, learn and improve.

I feel like I could have pushed this project even further, and that I made some mistakes along the way. But it’s also vital to finish what you started so you can assess your progress, take what you’ve learned, and move on to the next challenge. 

Have a goal, have a strategy, find a community of like-minded artists to inspire and help you, “trust the process”, keep on pushing yourself and finish what you started. Also, never stop studying: do study sketches, look for comprehensive references, and invest in good education programs that will help you. And don’t forget to enjoy the journey!

Milton Fernandes, 3D Character Artist

Interview conducted by Arti Sergeev