ArtEngine: Workflow & Features

Introduction

My name is Alix Coleno, and I’m a Technical artist primarily focused on modeling, texturing and scripting. Before working in the video game industry, I used to be a web developer. I left the field and started over my studies in game art in a French school named Aries (now Brassart). I then began my career in a French video game studio called Kylotonn, in Lyon, as an environment artist for the WRC franchise. After shipping a couple of games I pursued my journey as Lead 3D Artist before joining Unity a few months ago as a Technical Artist. 

I think I fit in the generalist label and my previous work covered environment, vehicles and surfaces authoring. I also focus part of my work on automation and procedural generation.

I’ve recently joined Unity Technologies on the Artomatix team in Dublin. My primary role is to help the development team understand the way artists are working. I help define the pain-points we will solve, outline the workflows, and ensure we will meet and exceed industry content creation standards with our tool ArtEngine.

ArtEngine

As my workmate Victor Kam explained in his 80.lv interview, ArtEngine is a node-based standalone app, working with any engine, focussing on the creation of materials. It utilizes an Example-based Workflow, meaning the user inputs data and the software can manipulate it to fit their needs.

With this approach, the software is versatile and helps eliminate some of the major pain-points faced by artists in their creative process when it comes to texture authoring (removing seams, creating infinite variation of a material, cleaning scans …).  I spend every day along with a team of artists, assisting developers on getting the most out of this very new approach to create top level quality content for every creator. No matter the industry, if you need to quickly create materials from photographs and scans, we want ArtEngine to help you reach the quality you aim for.

Artomatix and ArtEngine

The story of Artomatix and ArtEngine is mostly a story driven by years of research and development at the intersection of AI and Computer Graphics science, led by Dr. Eric Risser, the founder of Artomatix. Within Unity, Artomatix has recently transformed into the “Content Business Unity”, led by Joe Blake, the former CEO of Artomatix. The team has tripled in size, and continues to be passionate about pushing the boundaries in digital content creation.

We, the artist team in the Content Business Unit at Unity, are here to challenge the software we build and create high quality content to push forward the Example-based Workflow. If a sophisticated tool is made only by developers and researchers; it may be able to produce results never seen before, but  might not be very accessible to creators. We want to ensure the software stays innovative and cutting edge, but also intuitive and artist friendly.

One of Unity’s values of “users-first” is at the core of everything we do. We bring a users’ point of view to all those awesome researchers and developers to make the software accessible to all with the least technical background needed. We would like creators to have the freedom to focus on making art instead of fighting against the software.

ArtEngine's Features

Even if you can work a material out of a procedural workflow, the strength of ArtEngine resides in the Example-Based Workflow. If you give the software some input to work with, you drastically expand the range of the possibilities.

Along with all the standard tools you would expect from any image manipulation software (like Levels, Blend, Blur), ArtEngine offers many AI-assisted features to automate several processes, like Seam Removal, Mutation, Pattern Unwrap and Up-Res. Those tools allow creators to automate the less artistic parts of the texture authoring with the help of the AI giving them more time to focus on what matters the most.

They also make material authoring more accessible. You don’t have to be a trained Technical Artist, with deep domain expertise in every artist tool, to just pick an image of a surface you like and make it tileable and ready to use in any engine.

Making a Material Usable in a Game Engine

There are multiple ways to do that. You can just capture a nice material with your phone and bring it to ArtEngine and use all the generation nodes available.

If you aim for a higher quality, you can set up a bit of a rig setup, to capture a surface with varying lighting angles.  I’m currently using this second approach, with different rigs depending on the size of the surface we want to capture, and 8 different light sources.

After some calibration we perform that are specific to our rigs, we feed everything to the Multi Angle To Texture node.

Another approach would be to do a full photogrammetry capture with 3D reconstitution of your surface and use baked data to generate a material. Victor Kam made a nice video presenting an example of the process to perform this operation:

At the end of the month, we’ll be releasing A beginner’s guide to photogrammetry with Unity ArtEngine, so keep an eye out for that. 

Creating a Material From Captured Data

Let’s take a look at the project for the Blue Taransay Fabric showcased in this article. Here is the sample we used for it:

The first step we are taking is to take 8 different shots of the sample with 8 different light sources, evenly distributed around the sample with a custom rig we built. The original shots are 19120 x 12736 px wide but I cropped into them to remove unnecessary background and lighten the images for the software. Then we import those 8 images into ArtEngine and the Multi Angle To Texture node.

I then crop the area that I am actually interested in in the sample to create a material from. I did not perform this cropping before because I want to be able to change my mind at any time of my graph.

So far, because of the nature of the material, seams are already subtle, especially in the X axis but they are visible when we enable the tiled view. We will fix them later.

Next step is to generate all the material maps. Multi Angle to Texture can only generate Albedo and Normal maps. All the other maps are then deducted from those two.

As you can see, the generation of those maps is pretty straightforward. All the material I am generating from scans have a similar series of nodes to create all the maps we need for a PBR material. I just have to tweak settings on each node depending on the material I am creating and the whole graph is ready to go. 

Now that we have a full non tileable material, there is only a last step to do: removing the seam.

This part relies on the seam removal algorithm. You can perform it through a Mutation or directly using the Seam-Removal node, depending on what you are trying to achieve.

Given the nature of this material, a very defined pattern, tiling almost on its own, I decided to go for a Seam-Removal.

With the help of the AI, the seam is removed in all channels using a single node. Even if there is still a slight distortion on the Y seam, the seam is not brutal anymore. I can now choose to keep it as a natural disturbance of the pattern or I can correct the alignment upstream and re-perform the seam removal.

To finish, I just correct the size and output a full 8k material ready for any real time engine.

Processing the Scanned Material

I need roughly an hour to tweak all the settings to fit my needs for a new 8k material. But when this phase is done, the whole graph takes only 2min to compute so I can nearly automate the production of any similar material.

The time you will need to process a material depends on the size of your material as computing time is strongly impacted by the size you wish to process. So I tend to do a first pass work with smaller input to set all the settings and then perform the final computing with the bigger data.

I largely rely on “template” projects I have made to spare myself half of the settings on each processing.

Hardware Requirements

To run ArtEngine a Windows PC is required along with an NVidia card. The AI processing relies strictly on the GPU so more VRAM the better.

I am personally running with an RTX 3090 to process the heaviest scans.

I find it very thrilling to work with tools like this, powered by AI, that facilitate the creation of assets. Until recently, photogrammetry and scanned material necessitated a lot of logistics and expertise and most studios were condemned to buy the materials they wanted on content libraries. With AI, any creator can generate the material they want directly from their own environment in a very few time.

On the other hand, if a team finds something they like on a content library, they can now, with the help of the AI, produce infinite variation of the material they purchase to fit any of their needs. Creators are free of the principle downside of content libraries: being trapped using what you downloaded as you downloaded it. With this AI, this is history and any content library is now a vast collection of ideas and inspiration.

With those super easy to use tools, anyone can bring up their own part in content creation and facilitate the work of the artists in the team.
We are only at the beginning of what AI assisted content creation will be in the future and I am very excited to be part of all the improvements that are going on at this exact moment, here, at Artomatix. I hope you will be as interested as I am in those new techniques for material authoring!

Are ArtEngine Customers Still Largely AAA?

We do have a lot of AAA customers whom we have extremely close relationships with to build pipelines to fit their needs and acknowledge their feedback to make the product better. 
However, we are also building features and tools for the indies and are really excited to open ArtEngine up to them! We are offering a 30 days trial for anyone to try Artengine and see how it could fit in their workflow alongside a GDC promotion happening where the 12 month subscription of ArtEngine is available at a massive discount for the next two-month.

Here at Unity, we are excited to open up content creation to the most creators in the world, regardless of the size of the team and I believe that ArtEngine is this kind of tool.

Alix Coleno, Senior Technical Artist at Unity