80 Level Digest: Developing Apps & Games for Vision Pro

At the Worldwide Developers Conference in June, Apple unveiled its long-rumored AR headset Vision Pro. Developers and potential users alike were discussing it for the next week, from its sleek design and cool interaction system to, of course, its $3,499 price.

The device introduces its users to spatial computing. It is meant to be controlled as naturally as possible: with your eyes and hands. It seamlessly works with Apple apps and can be connected to a MacBook wirelessly.

Apple thinks this is the future, and if you agree and want to contribute to this glorious vision by making an app or a game for the headset, this article might help you. Here, we are going to cover the materials you will need to create your first program.

Soon after the Vision Pro announcement, Apple released the visionOS software development kit (SDK), which you need to have a developer account for. After this, you're ready to go!

The first page you should visit is Apple's visionOS development site. It has everything you need to know, including frameworks, tools, and a plan on what to do.

If it's too overwhelming at first, I'd recommend watching this video from Sean Allen, where he explains the concepts of windows, volumes, and spaces that are essential for Vision Pro. Windows are what we see, like a window of your browser, volumes take depth into account, so this is where the 3D objects live, and spaces combine the two: this is what you see around you, including your room with the Passthrough feature. 

Allen then talks about the best choice of framework for development, running existing iPad and iPhone apps, shares some design tips, and discusses privacy.

Here are the tools you'll need for your apps, all Apple-approved.

Xcode

"Development for visionOS starts with Xcode," Apple says. This is your go-to development environment where all the magic happens. "Add a visionOS target to your existing project or build an entirely new app. Iterate on your app in Xcode Previews. Interact with your app in the all-new visionOS simulator and explore various room layouts and lighting conditions. Create tests and visualizations to explore collisions, occlusions, and scene understanding for your spatial content."

SwiftUI

This is a development framework for building user interfaces that Apple recommends using instead of (or with) UIKit. "With all-new 3D capabilities and support for depth, gestures, effects, and immersive scene types, SwiftUI can help you build beautiful and compelling apps for Vision Pro."

RealityKit

RealityKit is integrated with SwiftUI "to help you build sharp, responsive, and volumetric interfaces." It will let you create AR experiences by providing 3D simulation and rendering capabilities. "RealityKit can automatically adjust to physical lighting conditions and cast shadows, open portals to a different world, build stunning visual effects, and so much more."

ARKit

Together with RealityKit, this tool allows you to work with the Vision Pro user's surroundings. It combines motion tracking, world tracking, scene understanding, and display conveniences.

Reality Composer Pro

And to make sure your 3D objects work right, Reality Composer Pro allows you to preview and prepare 3D content for your visionOS apps. It can help you import and organize assets and is available with Xcode.

Unity

This player is very familiar to our readers. You can use its tools like AR Foundation to create apps and games, which will have access to visionOS's features, such as Passthrough and Dynamically Foveated Rendering.

Unity offers a beta program on spatial creation "where you can create with our tutorials, templates, and sample apps. You can bring your existing Unity-created content to reimagine your experiences or create something entirely new." You need to register if you'd like to take it. I couldn't find the price of the course, so let me know if you have this information.

When you get yourself acquainted with the software, it's easiest to follow Apple's plan:

• Start the software development process with Xcode
• Add a new dimension to your interface
• Devise straightforward interactions
• Create next-level audio and video
• Embrace inclusion
• Prioritize privacy
• Test and tune your app

This page will help you get access to the documentation and educational videos you need. It covers the following topics:

• Get started with visionOS
• Meet spatial computing
• Explore SwiftUI and RealityKit
• Rediscover ARKit
• Design for visionOS
• Explore developer tools for visionOS
• Develop with Xcode
• Meet Reality Composer Pro
• Get started with Unity
• Learn about TestFlight and App Store Connect
• Build games and media experiences
• Build for collaboration, sharing, and productivity
• Build web experiences
• Run your iPad and iPhone apps in visionOS

Apple has three "must-watch" design videos that will help with your development journey. The first, "Principles of spatial design", teaches how to design with depth, scale, windows, and immersion and make comfortable, human-centered experiences.

"Design for spatial user interfaces" discusses UI components, materials, and typography that will make your app legible and easy to use. 

Finally, "Design for spatial input" covers how to design great interactions for eyes and hands, the design principles for spatial input, and best practices around input methods.

Spiel Times also has a great article pointing out what you need to do to make sure your project looks and feels good. Here are some of the tips:

To get a fuller understanding of the development process, you can check out these videos as well:

As the headset is not available for the public yet, there aren't a lot of materials discussing development. Hopefully, once it's out next year, we will find more to learn from.

Do you have a favorite source? Share it with us and don't forget to join our 80 Level Talent platform and our Telegram channel, follow us on Threads, Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.