Recreating the Drawing Mechanic from Half-Life: Alyx in Unity

Daniel Santalla explained how he set up the mechanic that allows the player to draw on a blackboard in VR and how it can be done with the glass surface similar to the effect in Half-Life: Alyx.

Introduction

My name is Daniel Santalla, I live in Merida, Mexico, and I work at Recreate Games as a Technical/VFX artist. I started my game dev journey when I was 16 as a 2D artist but have jumped between roles ever since, going through game design, audio, and programming to finally end up as a tech artist doing shaders and visual effects. I studied interactive design at the Universidad Modelo and at the same time I co-founded Cometa Games, where I developed mobile games like Pingo Park, Beast High, and Monkeynauts.

Today, I'm working on Party Animals as a tech artist with a lot of very talented people. Also, I usually create educational VFX content that I share on my Twitter page.

Thoughts on Unity

I think that the best thing about Unity for a tech artist like me is that it is a really hands-on engine. You have to build most of the systems yourself and this is the perfect way to spark creativity and create unique concepts.

Idea of the Project

I actually came up with the drawing mechanic after creating a similar effect that highlights a gameobject. All I’m doing here is using a distance node that calculates the distance between the gameobject’s position and the fragment’s world position.

This effect sparked the idea of having a shader that could read past positions of the object to simulate a trail. The obstacle here is that the shader graph doesn’t support vector arrays (yet) so I had to come up with a more elegant solution.

Setup

The way I addressed the mechanic is a bit hacky but is really easy to pull off, even when you don’t have a lot of knowledge about shaders.

The setup is using two key components: an orthographic camera and a render texture.

The first thing you want to do is to place the camera right in front of your canvas. With the help of culling masks, the camera will be able to render anything that gets in its range. Then, you’ll want to output the camera to the render texture.

Now as you can see, if we drop the render texture into the albedo of any material, we’ll be able to see only what’s inside of the camera’s reach. To finish this up, all we need to do is not clear the camera every frame. This is done in different ways depending on the render pipeline that you’re using. In URP, all you have to do is to set the background type of the camera to “uninitizialed’’.

At this point, you’ll be able to draw on the board anything you want. Using culling masks you can create a layer for your painter objects and create things like tips for markers, stamps, the tip of a pencil, etc. Here you can check the system working with a brush in a game I’m making called Art Studio VR:

Recreating a Similar Effect from Half-Life: Alyx

In Half-Life: Alyx, a noticeable part of the effect is that you’re not drawing on a blackboard, but instead you are painting over a transparent window. This is actually a really simple thing to implement into our workflow. All we have to do is replace the default Material with a custom shader that supports alpha tweaking and transparent rendering.

Here you can take a look at my graph where I also added some normal textures to make the window more realistic:

Pen Pressure Simulation

One thing you’ll notice when testing the mechanic is that you’ll naturally lean into the board and will lose immersion when you don’t feel physical push back on the pencil. This leads to an unnatural feeling of not being able to control the pen pressure. A way I’ve discovered to cheat this sensation is to use different tip shapes and rely on the natural motion of the hand instead of actually detecting the force pushing into the board.

Is Color Mixing Possible?

With this workflow, mixing colors is simply not possible, because the camera renders new frames on top of the others and there is no straightforward way to mix between them.

Afterword

If you have any questions or you came up with anything cool involving this technique, my Twitter DMs are open, please let me know!

To learn more about tech art, check my Twitter page.

Daniel Santalla, Technical/VFX Artist

Interview conducted by Arti Sergeev

Keep reading

You may find this article interesting

Floating Islands: Setting Up Stylized Shaders with HDRP in Unity