July 24, 2023

Windowing on the Vision Pro

Window management in visionOS is handled using familiar SwiftUI APIs, but with a few restrictions as well as opening up some new opportunities.

First thing’s first, if you’re extending an existing App, then you may need to Enable Multiple Windows support in the info.plist of your app. Adding the `UIApplicationSupportsMultipleScenes` with a value of true will unlock support (required on visionOS and iPadOS. Then it’s as simple as calling openWindow by specifying the id of your window group using the openWindow environment action.

Once you’ve done that, there’s seemingly no limit to the number of windows your app can open or where your users can put them.

struct MainWindowView: View {
    @Environment(\.openWindow) private var openWindow
    var body: some View {
        VStack {
            Text("Main Window").font(.title)
            Button("Open") {
                openWindow(id: "childWindow")
            }
        }
    }
}

Apple changed the way windowing works on Beta 2, allowing more control of window sizing. The below has been updated to reflect that.

In Apple’s attempt to control the user experience, and ensure the user isn’t disorientated, by default every window launched is the same size and front and centre of the user’s view. Initially you couldn’t override this functionality, but as of Beta 2 it is possible to add a `defaultSize` modifier to set the initial size of the window, but still not its position. It also doesn’t seem possible to adjust this programatically after the fact (without some very low level, and clearly unintended, trickery)

.defaultSize(width: 0.5, height: 0.75, depth: 0.0, in: .meters)

If you do need a specific and fixed size for your content, then one trick is to use a volumetric window, which does allow you to set a size. Volumetric windows are still opened in the middle of the user’s eyeliner, but can be moved around and placed, but not ever resized by the user.

Points in visionOS are calculated differently to other platforms, and don’t represent a fixed number of pixels. Instead a point on visionOS is an angle, which means content can dynamically resize as it moves closer or further away from you, in order to maintain legible content. The recommendation is to only fix the size of an object if it represents a real world object, of a specific size, otherwise it’s best to let the user control how they interact with your content.

The following code allows you to create a window with a fixed size, which can’t be controlled by the user. Despite this being a Volumetric window, it can still contain only ‘UIKit’ elements, the only difference seemingly is no-matter what the depth of the view is set to, it’s possible to move the window around from the ‘side’ which also re-orientates the view perpendicular to the user.

WindowGroup(id: "main") {
    MainWindow()
}
.windowStyle(.volumetric)
.defaultSize(width: 1, height: 1, depth: 0.1, in: .meters)

There’s no concept of a Window in visionOS, only WindowGroup. That means a user can open multiple copies of a window by default. This may be intentional, as I imagine it could get easy to lose windows in your environment, so having the ability to create them at will means the user can control this and even leave multiple copies of windows around their environment.

If you want to ensure only one copy of your window group, then the easiest way to manage that, is to dismiss the WindowGroup right before you open it. This will mean that it re-opens front and centre where the user is looking. This could of course be a benefit, in case they’ve misplaced the window somewhere.

Button("open") {
    dismissWindow(id: "targetView")
    openWindow(id: "targetView")
}

If you don’t want windows jumping around, then you’ll need to keep track of which views are open and dismissed using a combination of Scene Phases and/or `onAppear`, `onDisappear` modifiers to manually control the ability for the user to open a new instance of that WindowGroup. Scene Phase doesn’t report when a window is dismissed, either using dismissWindow or using the visionOS close button that’s attached to every window. It also doesn’t support the `controlActiveState` environment variable, which is a a macOS solution to get this information.

WindowGroup("Target View", id: "targetView") {
    TargetView()
        .onAppear {
            print("target window appeared")
        }
        .onDisappear {
            print("target window disappeared")
        }
}

struct TargetView: View {
    @Environment(\.scenePhase) private var scenePhase
    var body: some View {
        Text("Target View")
            .onChange(of: scenePhase, { _, phase in
                print("scene changed: \(phase)")
            })
    }

}

Immersion

If you need more than just windows then the Immersive space is for you. Here you get ultimate control of where objects are placed and aren’t restricted to a window at all (although windows still work here too). Even if you do have an app that suits an immersive experience, it’s still recommended to launch into a regular window to start with, using say a button within that window to open an immersive space under the control of the user. Perhaps if you have a true immersive experience or game, then after the first time, subsequent launches can go straight to the action if appropriate. To open directly into an immersive space requires setting another plist value, this time `UIApplicationPreferredDefaultSceneSessionRole` needs a value of `UISceneSessionRoleImmersiveSpaceApplication`.

Note: It doesn’t currently seem possible to open an immersive space upon app launch in simulator. You do need to specify a key in the info.plist to allow this in the first place, but in my experience, the immersive view isn’t rendered. In Beta 2 it seems this key is ignored and the app refuses to launch.

It’s worth knowing, that currently the original window group remains open when opening an immersive space. This may not be desirable for your experience, it certainly feels a bit redundant in some circumstances, even if the user can always manually close this window. This does reveal a weird UX issue though, as if the user closes the initial window, then also closes the immersive space, then your app may have no windows left, so it will be fully closed.

If you don’t want this behaviour then you can resort to managing the opening and closing window groups manually. Apple handles this weirdly in the Earth sample app, by using opacity to hide the main view, replacing it with a smaller, space specific modal. I’m not a fan of this approach as if you move that view out the way, the main view is confusingly (to the user) also moved. From a design perspective, if the new view looks and acts like a different window, it should be treated as such. My preference is to open a separate window group, or use the attachments option of the RealityView to position the modal view within the scene itself.

In the main window, open the immersive space as normal:
await openImmersiveSpace(id: "ImmersiveSpace")

When the immersive space opens, dismiss the original window:
.onAppear {
    dismissWindow(id: "mainView")
}

When you want to exit out of the immersive space:

//re-open the main window
openWindow(id: "mainView")

//if we don't wait here, the app doesn't seem to have time to open the window
DispatchQueue.main.asyncAfter(deadline: .now() + 0.1) {
    Task {
        //then dismiss the immersive space
        await dismissImmersiveSpace()
    }
}

A window into the future

As developers explore the options with windows, then I expect certain paradigms to become the standard, perhaps with some changes to visionOS as the beta period continues. One such limitation with visionOS Beta1, is that there is no access to the Digital Crown, which means there’s no way to test the progressive immersive space, in which the user controls the amount of immersion of the space, even going from .mixed through to the .full display options as they desire. This has now been resolved as of visionOS 1.0 Beta 2, it’s now possible to set the immersion values to 30, 50 or 100% via the I/O menu.

I also haven’t touched upon opening windows onto surfaces, re-orientating windows so they’re flat on a floor/table, or portals. I feel like a part 2 is coming…