From Beauty to Beast: Face Transformation with Blendshapes

Introduction

Hi, my name is Rene Vidra and I live close to Vienna, Austria. I’m Co-Founder of the CG-company Utopiatree, working in the fields of illustration, film and Augmented Reality.

Career

I started my career as a CG Artist in 2009 after graduating in Interactive Entertainment Design at the SAE Institute in Vienna. Back then, I had been working as a freelancer for several clients and projects, mostly in the fields of 3D Projection Mappings, Advertisement and Augmented Reality. Though I initially wanted to work as a concept artist, I discovered my passion for 3D very early.

For most of the projects, I was working as a generalist and functioned as a ‘problem-solving guy’, although I am specialized in rigging and animation.

Passion for Character Creation

I really love every aspect of creating a character but I didn’t have the opportunity to make them professionally yet, though I’m mainly interested in that field and want to improve my skills in that direction.

That was my main reason for taking CGMA's course Character Creation for Film/Cinematics with Pete Zoppi.

The workshop helped me a lot with my current job at that time, working as a Head Instructor for Game Art at the SAE Institue. I also participated in a similar workshop in 2015 with Sze Jones. It was a blast and I improved a lot during that time.

Mermaid: Initial Idea and Collecting References

There are actually a couple of reasons why I chose a mermaid for this course. Pete Zoppi recommended doing only a bust of a character for his workshop due to the amount of workload and the limited time. By creating a Mermaid, I felt pretty comfortable doing the whole character because I didn’t have to do any clothing. I also like the challenge of creating a really appealing and pretty female character.

The idea of doing a beast transformation came to my mind while collecting references and reading about the myths of mermaids.

It is very important to make your character outstanding and believable by giving it a short story beforehand instead of making all the decisions throughout the modeling and sculpting process. In my case, I wanted to have a young and innocent mermaid that turns into a wild beast the moment she is getting attacked or trapped.

By creating a reference sheet instead of just saving hundreds of different reference images, you can clarify in which direction you want to go and then stick to it.

It also helps to create several reference sheets for the different aspects of your character like clothing, haircut, specific body parts, etc.

Modeling

To be able to rig and animate the face later on in Maya, it is important to have a clean topology. To save some time for the design process itself I started with a base mesh I’ve done before. I deleted the legs and modeled the base for a typical mermaid tail and added some fins for her upper body. It was a pretty straightforward process without any major complications. The hardest part was getting a feeling for an interesting shape and silhouette. A simple way to check the form of your character is to turn off the light in the 3D scene so you see only the silhouette and then rotate the camera around the character.

UDIMs are essential to get enough texture resolution. For the Mermaid, I decided to use 5 UDIMs, separated in the face, upper body, arms, tail, and fins.

Focusing on the Beauty of the Face

I decided to spend a lot of time on her face, focusing on the transformation into the beast. The main goal was to achieve a super-realistic face without losing a touch of style. I always tried to keep some kind of softness and saturation within the skin and texture.

In ZBrush, I mostly used standard brushes like the move tool to work on the general shape and of course, the DamStandard Brush to sculpt the main features of the face.

An important step to get realistic skin these days is to get sources of references and high-detail textures like the ones from texturing.xyz. I got one set from their website following Pete’s recommendation to use the ZWrap plugin workflow for projecting a plane to the face and bake the textures later on in xNormal - an awesome process to get highly detailed information to the face. You can check out this workflow in the tutorial section here.

Making the Transition between Body and Tail

Projecting reference photos onto the upper body could be done with several software packages like Mudbox, Mari, Substance Painter, and Spotlight within ZBrush. All of them do a pretty decent job, though every software has its strengths and weaknesses. In my case, I mainly used Spotlight to stay in one program.

One of the toughest parts of the production was her tail, as well as blending it with the upper body. I had several approaches to sculpting it in ZBrush. Unfortunately, I wasn’t happy with any of the results because I wanted to have a non-destructive solution to always be able to change the size of the mermaid’s scales afterward.

Therefore, I decided to solve it with texturing in Substance Painter. I chose a material called “Creature Scales” that I found online on Substance Share to get exactly the look that I wanted. For getting the best results, I created two layers and projected the material on the front and side via tri-planar projection. Afterward, I went on with blending those projections by using a Mask.

It took some time to get a good sense of the different sizes and the right transparency level of the scale layers, but it was an important factor for a good end result. After I grouped all of the different layers together, I created a final mask to paint in a smooth transition with a big soft brush.

Creation of the Teeth

To achieve realism for the face and later on for the beast mode, I also wanted to focus on the teeth to get the same level of quality as the skin texture.

My approach started with a texture projection. I did the sculpting afterward to have a general idea which brushes I could use and where I would add some details. In production, this method can save you a lot of time.

After building up a proper shader in V-Ray, it took some time and many renderings to get a nice result. Especially finding a good value for subsurface scattering is very important for a realistic look.

From Beauty to Beast: Using Blendshapes

My experience as a rigger over the last years helped me a lot to find a solution for the transition between the two states of the mermaid. I knew that a blendshape would be the easiest and fastest way to get the result I was looking for, due to the fact that blendshapes are just checking the movement of every vertex.

Usually, characters have some kind of clothing and, therefore, a separate head that can easily be duplicated for blendshapes. In this case, the mermaid was one single mesh, without any separations. Duplicating the whole mesh for every blendshape would cost a lot of performance and increase the file size. The proper solution was to separate the head from the body just for a moment. Then select the head and then the body. By combining them back together and merging the vertices, every vertex gets a new number, beginning with the vertices of the head. Now, I was able to duplicate the whole mesh and delete everything except the head without reordering the vertex numbers for the blendshape.

I exported the blendshape to ZBrush to modify and achieve the beast face. Using reference photos of angry people and studying the changes in their faces helped me a lot to get the final results.

Back in Maya, I added a blendshape node to the original Mermaid mesh and connected the Beast blendshape to it to make the transition possible. It is important to uncheck ‘check topology’ in the settings, due to the deleted face of the blendshape. 

Adapting Textures and Shaders for the Beast Mode

To make the transformation really strong, I decided to take a different diffuse texture, add a blendshape for the teeth, and change the shader of the eyes.

I used Substance Painter again and utilized an HSV filter to adjust the saturation and hue value of my diffuse texture. Adding some extra features like veins and darker eye shadows helped a lot to make that beast texture really badass.

For the teeth, I went with the same approach and created a blendshape as well as new textures for the beast state.

To get dark eyeballs, I simply modified the refraction value of the shader for the outer eyeball geometry and changed the diffuse value to black.

Connecting Everything Together via Driven Keys

Now it was time to combine everything together and to create a simple control for the whole transformation. I used a simple NurbsCircle as a controller and built up a ‘range slider’ next to the mermaid’s face to control everything at once.

By using driven keys, I connected the blendshapes of the face and teeth to the NurbsController. The same procedure goes for the refraction value of the eyeball shader. The blending part of the textures was a bit more complex. By adding a BlendColor Node in the Node Editor, I could easily add another set of textures for the Mermaid's diffuse and displacement. Connecting that BlendColor node into the driven key system finished the whole setup.

Now, it was possible to pull the controller to the right and transform her into a beast. Of course, for a real facial rig, you would need to create many more blendshapes and controls, but for the workshop, I wanted to show the current progress and keep it as simple as possible.

Presentation: Preparing Final Renders

I already spent a ton of work on the character, so I wanted to do more than just some simple T-pose renderings. As a rigger, I felt it important to build up a complete rig that I could use for some animations later on. Finding an interesting pose and composition as well as doing a proper lighting setup can be a bit tedious. It is fascinating how much a character can change within another scene and lighting setup. Using some references again to choose a strong pose and composition led me to the following result. After rendering the final image, I did some post-production by adjusting the saturation, levels and adding minor details in Photoshop to achieve the final look.

CGMA Course: Feedback

I definitely recommend the course to other students. Pete Zoppi is an awesome instructor and did a great job by improving everyone’s work while keeping the motivation very high. His live feedback sessions were not only supportive but also inspiring to take your character to the next level. 

Rene Vidra, CG Artist & Co-Founder of Utopiatree

Interview conducted by Ellie Harisova