Skyboxes and transparency in VisionOS
I was recently experimenting with adding a glowing gradient material to a cylinder in RealityKit for use on the Vision Pro. This led me down a journey that crossed meshes, materials, Blender, Reality Composer Pro and a bug in Apple’s rendering.
The first stumbling block was having no cylinder mesh resource in RealityKit, like there is in SceneKit. For “simple” things I prefer to generate meshes and materials in code where possible. It keeps app sizes down, and keeps the asset workflow simpler.
However in this case, it was necessary to rely on Reality Composer Pro, which does have a Cylinder geometry and opened up the potential to explore the shader graph feature for the first time.
With a bit of research and some trial and error I was able to create a nice pixel shader effect that uses the height of the pixel to determine the transparency, along with an added remap node to ensure the bottom of the cylinder (< 0.2 in y) is also transparent. With a bit of experimenting and reversing the gradient with a -1 constant multiplication for the y position, local to the mesh I had the gradient “glow” I was looking for.
This was the effect I was after, but I couldn’t uncover a way to make the material double sided. I couldn’t see any options for the unlit surface I thought was appropriate, nor, in my desperation figure out a vertex shader that would achieve the effect.
Almost simultaneously there was a discussion on Mastodon about the lack of an out-the-box skybox for VisionOS. In Apple’s sample code, they use a trick of negative scaling the entity to create a skybox. This is certainly one solution, but isn’t possible in Reality Composer Pro as negative scales aren’t possible. This means you can only visualise the result within Xcode (either the preview or Simulator).
//scale a mesh entity to reverse the surface rendering
.scale *= .init(x: -1, y: 1, z: 1)It’s also not a solution for my situation as I needed a double sided material, not just an inverted one. One way around this is to duplicate the entity and only invert one, again within code. This does solve the problem, but I wasn’t satisfied with it as a solution, as it just feels wasteful and relies on a runtime fix. Online I found a few other options, one being to duplicate just the mesh, rewriting the vertices in reverse order. This certainly inverts the face rendering, but without testing, my gut feel this would be more expensive than duplicating the entity itself.
Going back to experimenting, I discovered that my original choice of material, ‘unlit’ may have been hampering my options. Two other materials, ‘Custom’ and ‘PhysicallyBasedMaterial’ have a ‘faceCulling’ option, which is described by Apple as:
To improve performance, RealityKit culls polygons, or faces, that it determines won’t be visible. Discarding faces that aren’t part of the final render eliminates the need to do any calculations for those faces.
The three options for culling are ‘back’ (default), ‘front’, or ‘none’. Bingo, this is exactly what we needed. This serves the purpose for both what I needed (.none) and the requirement for a skybox (.front), without needing to scale the entity.
There are two issues with this approach. There’s added complexity (and potential performance implications) of these shaders over the ‘unlit’ I’d been using. My assumption would be the simplicity of unlit, allows for some performance gains under the hood over a custom or physics based shader. This could certainly do with some additional investigation, but is a small problem compared to the additional issue of CustomMaterial’s weirdly not yet being available for VisionOS, so aren’t even an option here if shaders are part of the rendering. It’s still a great solution for a Skybox situation, using a PhysicallyBasedMaterial, but isn’t possible yet using a Reality Composer Pro workflow (no access to any parameters for a `ShaderGraphMaterial`).
One thing to note, is there’s an issue (FB12605220) with RealityKit when using blending and no face culling, that causes some erroneous culling still as shown below in the left image.
A fix is to specify a value for ‘opacityThreshold’ on the material, even if it’s just the default of 0.0. This seems to kick the rendering into gear and it then correctly renders the transparency with no-culling of “hidden” faces.