Art Studio VR: Developing a Painting Simulator for VR

Introduction

Hey everyone, I’m Daniel Santalla, I’m an indie Game Developer with a focus on graphics and VFX and currently, I’m developing Art Studio VR, an artistic painting experience made for the Oculus Quest. I started developing this game about a year ago with Ana Bolio. When I’m not developing this game, I’m doing shader and VFX tutorials for Twitter or working as a tech artist at SMG studio.

Art Studio VR

I love VR and I really think everyone should try it at least once. I also love art in general and I always try out new software just to see what I can do with it. I guess you could say that the idea of making an art app was something that I always wanted to do, the VR part appeared by constant experimentation. It wasn’t supposed to be VR, but after adding the headset support, I couldn’t go back, it was too cool!

Technique

The original prototype used a technique that involved having a second camera facing the canvas only seeing the tips of the markers and not clearing the frames (you can do that by setting the camera to “not clear flags” or “uninitialized” mode). Then, I saved the result to a render texture and used that as an albedo in the canvas. It was really straightforward and easy to do. The issue with it is that the “not clear flags” method is unsupported in Android devices so it was incompatible with the Oculus Quest.

New Approach

Currently, we moved away from the camera method and now we’re using a ray casting system. Basically, we use the hit position to know which part of the mesh was touched and then match that to paint a series of dots over the UVs. There are a few limitations with this method, for example, we need to have correct UVs in all the meshes we want to paint. Also, the seams in the UVs can be noticeable sometimes. 

Color Mixing

The color mixing is an additive workflow. Depending on the opacity of the colors, they will cover the canvas in a 0 to 1 range. So if you have blue color with an opacity of 0.5 and red color with an opacity of 0.5, the 1 value will be a mix of both colors. This is also true for the thickness and stroke direction of the paint, that’s done with Normal Maps, and each brush adds to a normal texture depending on its opacity.

VR Interface

I tried to make everything physical. After playing Vacation Simulator on my Oculus Quest, I came to the conclusion that they really nailed the interactions and made everything feel really fun, so I took some inspiration from them and continued on this no-buttons-needed design path for everything in the game.
 
In VR games, especially in those that involve using tools, you usually have diegetic menus attached to your wrists or hidden behind combinations of buttons. This trend is great for quick access to items like guns in action games, but I wanted to design everything in a way that non-gamers could easily understand. Tools are hung on the wall, floating menus have big buttons to physically press, and in general, everything has levers and handles to interact with. Here’s an early concept of the layout.

As you can see, everything is at reach. Want to use a tool? Just grab it.

With this design philosophy, a new set of challenges arrives, especially with accessibility. For example, what happens if you drop a brush to the ground but you can’t bend to grab it? Or what if you’re too short to reach the last bucket of paint on the top shelf? To solve this, we designed a set of rules that all the physical objects follow. For example, when an object touches the ground, it will wait a few seconds and then it will respawn near you but on a shelf or a table, this way, players never need to bend to grab something. Here’s how the layout looks so far:

Future Plans

The app is in really early stages. It’s going to be in continuous development, and the plan is to add other types of tools, not just oil paint and brushes. We’re experimenting with rock chiseling, stamps, paint guns, and other fun ideas that I’ll share soon on my Twitter.
 
Multiplayer is definitely on the road map, NFTs are there too. I guess it all depends on how the game evolves over time and how the VR community responds to it.
 
If you want to know more about the project, you can go to our website or contact me directly through Twitter, I’d be happy to hear your thoughts!

Daniel Santalla, Game Developer

Interview conducted by Arti Sergeev