LogInLight Brick Studio's Technical Lead Mikkel Fredborg discussed the story and development of LEGO Builder's Journey, spoke about what makes Apple Vision Pro different from other headsets, and provided a detailed overview of Unity's support for visionOS.
Shortly after Apple's launch of its tricked-out AR/VR headset, Apple Vision Pro, back in early February, Unity Technologies announced that its real-time 3D engine got full support for the headset's visionOS. This enabled Unity Pro, Enterprise, and Industry subscribers to leverage the engine's authoring workflows, built-in XR tools, and cross-platform compatibility to develop AR/VR experiences for Apple's new headset.
Prior to becoming generally accessible, Unity's support for visionOS was available in beta for a select group of developers across a variety of industries. One such developer was Light Brick Studio, the creator of LEGO Builder's Journey, an atmospheric, geometric puzzle game where you need to utilize various LEGO pieces in order to complete challenges. Leveraging the toolset and the templates, samples, and other resources it provides, the team easily ported Builder's Journey to Apple Vision Pro, describing the porting process as an incredible experience.
To learn more about the development process behind the visionOS version of the game, we spoke to Light Brick Studio's Technical Lead Mikkel Fredborg, who discussed the studio's history and their experience with Unity, shared some insights on LEGO Builder's Journey's development, and provided a comprehensive overview of the new pipeline the engine offers.
Discussing the history of the studio, Mikkel explained that Light Brick started as a small experiment with a mission to explore games that put playing with LEGO bricks at the core.
"With LEGO Builder's Journey, We wanted to deliver a heartwarming poetic experience that showcased this creative play material in the most beautiful way possible with an innovative yet intuitive game design. We're now 20+ passionate game developers working very iteratively with a combination of developer-driven ideas and frequent audience input to push the medium to fresh and playful levels."
When asked about the Apple Vision Pro headset itself, the developer called it an amazing piece of hardware that allows for a very wide range of experiences, noting that it is difficult to compare it to other headsets because of how radically different in design it is.
"LEGO Builder's Journey is using Bounded mode (AR), which is perhaps easiest to compare to a windowed app on a computer. This is both the most restrictive mode in terms of what you can do, but also the one that integrates most seamlessly with the OS.
There's also Unbounded mode (also AR) which is closer to a full-screen app, here you get access to more features in terms of hand tracking, feature tracking, scene tracking, etc. In both Bounded and Unbounded modes, the actual rendering and user interaction are handled on the OS level, so Unity does not render anything and can only receive a limited range of input from the OS. This makes it very different from other platforms.
Finally, there's also Fully Immersive mode (VR) where there's no passthrough video, and it operates more like a regular VR headset where Unity can render things directly via its Metal backend."
Speaking about the team's experience with Unity, the author revealed that all the projects they are currently working on are Unity-based, and they've been using it for other projects before developing Light Brick as well, so when it came to selecting the engine, it was an easy choice.
"It is fairly accessible and it is quick to prototype things in Unity. When you start scaling to bigger projects it takes some care and consideration, but most things can be done in an iterative manner because it has a modular design."
As mentioned in the intro, the Light Brick has had a chance to experience support for visionOS in beta, allowing them to have the experience very few currently possess. According to Mikkel, they've been working with the teams from Unity and Apple throughout the beta period. Since beta, the system has come a long way, enabling the devs to use many of the unique visionOS features directly from Unity.
"One of the big things to understand about the platform is that it is very much focused on privacy, so that means what we can access as developers is very restricted. In Bounded mode, we generally don't know where the player is, where their hands are, or where they are looking. Once the player starts an interaction by pinching their fingers, we start getting data about the position/rotation of the hand, and where the eyes looked when the interaction was started. This means there are a lot of design considerations we need to take when designing the gameplay, but thankfully, our game fits well within these boundaries.
Another huge difference from other platforms is that it is not Unity but rather visionOS/RealityKit that renders the game. As we have been working on custom versions of both LWRP and HDRP previously, we had to redo all rendering with baseline URP. Early in Unity's visionOS beta program, many features were not supported until shortly before launch. As a result, we had to go through various iterations and redo our shaders in ShaderGraph for them to work on visionOS in preparation for our launch.
There are also some specific differences between how things are handled in RealityKit and Unity, which require some work. When building for visionOS, all shaders are translated to the MaterialX format, which RealityKit uses, and we actually do some automated patching of the converted shaders in order to improve performance by eliminating duplicate code, etc. As RealityKit doesn't support every render feature that Unity can do, we also need special consideration for transparent materials, real-time lights, fog, etc., which all require some special handling both in shaders and game code.
Finally, we've rebuilt the game UI from the ground up for it to work properly on visionOS. Luckily, we don't have a lot of UI, so it was fairly simple to do. Overall I think there is definitely room for improvement, but from a game design and development point of view, once you understand the platform, I think it offers a very strong starting point for building outstanding experiences."
When asked about LEGO Builder's Journey, its history and development, and the role Unity played in the production process, Mikkel had this to say:
"LEGO Builder's Journey originally launched as an Apple Arcade title, so we have been working with Apple since starting the studio. They approached us asking if we were interested in doing some experiments with a new platform, and that eventually led to the visionOS version of the game. We have been working closely with the Apple Arcade team on getting the game ready for launch on visionOS, and it's been a really fun experience.
We have used Unity for building LEGO Builder's Journey for iOS, tvOS, macOS, PC/Steam/Epic, PS4, PS5, Xbox One, Xbox Series, Nintendo Switch, and finally visionOS. The visionOS version has definitely been the most radically different, but it has worked well for all platforms so far. Up until the visionOS version all the various platforms could be built from the same codebase, but now we have a little cleaning up to do, in order to get the visionOS changes integrated with the rest of the platforms."
In conclusion, the developer disclosed that the team is working on a number of new and exciting projects, while also improving LEGO Builder's Journey on visionOS, and shared some advice for aspiring creators looking to create a spatial experience for Apple Vision Pro with Unity.
Using a Mac with Unity, Xcode, and Simulator installed, it is possible to get started on experiments without a headset. A good place to start is by looking through the Unity discussion forums and trying out the various samples included in the PolySpatial package.
Mikkel Fredborg, Technical Lead at Light Brick Studio
Interview conducted by Theodore McKenzie
Keep reading
You may find these articles interesting