Metamorphosis: Making a CG Short Film in UE4

Introduction

Hi, I am Xi Wang (Jake), and I am a 3D artist. I came from a traditional fine arts background before I made the transition to the digital arts. 

I was an independent artist for eight years, doing exhibitions, selling paintings to galleries and museums, making toys, as well as doing animation. Then I began to learn 3D animation and visual effects at USC in the animation school. Learning and making digital art keeps me excited and hungry, with new ideas constantly popping up.

Since I started doing animation, I've been involved in many different projects, including VFX film, commercials, music documentaries, projection mapping shows, installations, and promotional events for film, independent CG animations. For example, J. Paul Getty Museum’s award-winning events Getty Unshuttered 1 & 2.0, The 3rd Annual Shorty Social Good Awards, 2019 World Changing Ideas). In addition,  several of my own CG films have also been selected and received awards in film festivals. I just finished my thesis film. I am working on a part of Getty Unshuttered 3.0 currently and discussing several projects. 

Learning Houdini

My school provided a Houdini course taught by Houdini expert Jordan Halsey. From my experience, in order to learn Houdini as a newcomer, some advice would be to take a class, watch online tutorials every morning, join the Houdini community, and use Houdini in personal work and projects. In addition to those daily exercises, you should also take the time to create and complete long term projects.

Combining Multiple Art Fields

This is the most interesting part. First of all, I was greatly influenced by Michael Patterson and Jordan Halsey who were my mentors and VFX teachers. Secondly, I am a visual artist. For me, both Motion Graphics and VFX are making beautiful images. As long as you have the patience to study, listen, force yourself to practice, you will gradually understand their principles and rules. I took Motion Graphics and Visual Music classes from Michael and Houdini classes from Jordan. So basically I combined my teacher's guidance with self-study. I went to art high school and studied painting and sculpture. During my undergraduate study, I majored in oil painting at Sichuan Fine Arts Institute, where I did some installation, painting, video art, animation, and exploring new media arts.  I just recently received my Master's degree in Animation & Digital at John C. Hench Division of Animation & Digital Arts, School of Cinematic Arts, University of Southern California. 

Metamorphosis: Inspiration

The inspirations come from a BBC story “I fixed a butterfly's broken wing to remember my mother”, a classic philosophical story “Zhuangzi dreamt of being a butterfly“ (along with the Tao Te Ching — one of the two foundational texts of Taoism), and a Chinese legend of a tragic story called “Butterfly lovers”. These three different stories have one thing in common - the butterflies which are often used as metaphors for change and hope, something ephemeral and beautiful which leads to a mixture of feeling like longing and sadness. I had this feeling when visiting Suzhou Gardens in China, it is so beautiful and already became a heritage. 

In the film, the core idea is the humanoid butterflies, ghosts of the lost world that tell the story of a beautiful ruin. For visual references, butterfly figures refer to Damien Hirst's butterfly paintings and Ridley Scott's “The Seven Worlds”. The desert environment design mixes “Blade Runner 2049” with Crescent Lake in Dunhuang, China. The dance performance refers to Bjork's “The Gate”. The musical reference was Ryuichi Sakamoto's "Life, Life".

Mutant Deer

In order to make the mutant deer as photo-realistic as possible, animal anatomy is a good reference to show the real details. The modeling of deer also integrates the painting and sculpture styles of deer in The Tang Dynasty of China, so it is not a completely natural animal, but a horrible and beautiful surrealistic creature.

The model is divided into two parts: skeleton and body. First, a healthy body is sculpted, from low poly to high poly. Then, bones are put in and used to identify rotting sites and carve out details. I separated the head from the body, allowing more detail on the head, with a low poly body. I then baked the displacement to this low poly model. The whole sculpting and bake process was done in ZBrush, and then I painted the textures in Substance Painter. I used a lot of textures and materials from Substance Source for blending and toning. Once finalized, I exported them as 8K SSS textures for use in Unreal. 

Animation

First, I made the pre-vis character animation in Maya, imported it into the Unreal Sequencer for animation blending, and animated the movements. Then I added more details to each of the animations in Maya and re-imported everything into Unreal to replace the previous versions. Also, I used Maya to adjust the motion capture details of the dance. It's mainly Maya, Motion Builder, and Unreal Sequencer, which is a standard real-time animation workflow. 

Environments

The first abandoned environment reveals an apocalyptic world in an uncertain time, a dead civilization. I wanted to create the image of the butterfly appearing in a limited space which contrasts with the open world that follows. In order to make the world believable, I used assets and textures from Megascans to add as much detail as possible.

Furthermore, the setting of Chinese screens and paintings symbolizes the characteristics of oriental civilization. The spider's web can indicate that human beings have not been present for many years. The god rays also helped to create the mysterious atmosphere of the ruins

The hot desert and the cave create a contrast in the color tone and at the same time they both have the sense of death; yet, the melted water in the cave brings the sense of life. I selected a lot of pre-made assets to build the scene, recombining and changing the textures to fit the concept of the film. This saved me a lot of time and allowed me to focus on creating unique assets such as damaged stone bridges, double deer, desert terrain, etc. 

Lighting

In each scene, sunlight is the key to determining the overall tone. The main light source of the first scene is the yellow volumetric light (6.0 lux), which gives the whole scene a brown tint. I used a spotlight spot where sunlight passes through the roof, enhancing the central floor brightness, and then used a rectangular light to give the room a little soft light behind the camera.

In the desert scene, the angle of the sun is low and orange (4.0 Lux) to help depict the apocalyptic atmosphere of the sunset. A spotlight was at the entrance to the garden. Two spotlights were centered around the deer and a rectangular light was to the left of the cave entrance.

In the ice cave scene, the volumetric sunlight (4.0 Lux) is blue, with a spotlight heightening the light on the ceiling. A point light is lighting the main stage. Each of the other entrances has a volumetric spotlight. There are 35 spotlights distributed along the camera's path to enhance the ice's reflections. There are also 10 spotlights surrounding the humanoid butterflies. In Level 2, the water cave, the sunlight turns to yellow. 

Camera Work

The original plan was that photographers would shoot the dance with the virtual camera at USC School so that we could create a handheld feel. Unfortunately, due to the impact of the epidemic, all cameras were animated by traditional keyframing. Sometimes the camera follows the butterfly, sometimes it explores on its own, like Alice's relationship with the white rabbit. One thing I learned was to not change the speed of the camera too drastically, it will make the movements seem smooth. At the same time, I got a lot of expert advice and kept adjusting again and again until it felt right. The rule about seamless change is to keep the camera's direction and speed consistent between the end of the previous scene and the beginning of the next.

Butterflies Simulation

The hierarchy of butterflies is important. There are small butterflies and large butterflies. The size of the butterflies in each shot should be adjusted. The butterflies should also move properly in the human form, not too fast or too slow. The wings of the butterflies use two-sided foliage material, which looks transparent just like the real butterfly wings. Jordan Halsey and I created the human-shaped butterfly simulation using Houdini. We built a particle-based system based on original mesh and animation from the mo-cap data. I then used a Copy to Points node to form the particles and instance our butterflies to the points. In addition, their flying is also part of the simulation. For example, the butterfly gathering into a human shape is a reversal of the blowout effect. There was constant adjustment of the angle and force of the fan until I found the right feeling.

Challenges

The biggest challenge was the lighting on the humanoid butterflies,- it took a lot of time to adjust because thousands of butterflies have different surfaces and it is much more complicated than lighting the normal human skin. However, I used all imaginable methods, mainly animating many spotlights to follow the dance movements.

Overall, I want to continue to explore and learn about real-time CG. There are many potentially interesting things to do, such as Real-time Motion Graphic, Previs, Virtual production, etc.

Xi Wang, 3D Artist

Interview conducted by Ellie Harisova