A Surface to stand on

Welcome back to the Developing Series newsletter, where I share progress, challenges, and discoveries in building v2 of Series Photos App.

Last time I talked about getting a Mac app bootstrapped. With that out of the way and some basic system integration working, it's time to go deeper on what makes Series, Series.

I struggled with what to write about here, as I’ve just done a few rounds of complex work on the layout system. It's hard to come up for air and explain this stuff, but that's what this newsletter is here to force me to do…

So let’s start with the bottom of the stack. A custom layout and rendering library called Surface.

Surface

Surface provides the building blocks of a design app:

1. Responsive Layout System describing the relative size and position of elements. Responsive means that it adapts naturally to changes of output size and shape. This is how Series lets you change the Frame Ratio and everything just flows.

2. Fills to Draw colors and images. These are the things you actually see. We can add more such as gradients, text and video. Plus effects like opacity and blend modes.

3. Renderer for turning a description of layout into something visible. Surface currently renders to images (for export) and SwiftUI (for the app).

4. Introspection. To build an interactive design tool you need to know what part of the design a user is interacting with. Surface lets you tag meaningful elements and then inspect the model to find out what a user is interacting with.

Defining Layouts

Surface's approach to layout is inspired by SwiftUI (SwiftUI is Apple's latest tool for creating user interfaces). Having worked with SwiftUI quite a bit now, I realized that the way it describes layouts matches Series quite well.

Like SwiftUI, a layout describes a set of nested rectangles and any layout can be contained by any other layout. Each nested layer can adapt based on its outer container and inner content. From this you can define all manner of alignment and sizing primitives.

Here’s a simple example: a container that insets its content by 10 on all sides, with a red color fill inside. The container will expand to whatever it’s placed inside of.

FixedInsetContainer(
    insets: FixedInsets(uniform: 10),
    content: ColorSurface(color: .red)
)

Want a 4:5 ratio rectangle with that inside? Easy.

RatioContainer(
    mode: .fit,
    ratio: 4/5,
    content: FixedInsetContainer(
        insets: FixedInsets(uniform: 10),
        content: ColorSurface(color: .red)
    )
)

And so on. We can build up any number of containers to describe the layouts that Series will create.

Calculating Layouts

Calculating layout is based on SwiftUI’s algorithm. It's brilliant in its simplicity:

1. Parent Proposes Size for Child

2. Child Chooses its Size

3. Parent Places Child in Parent’s Coordinate Space

Let’s apply that to our very simple example.

FixedInsetContainer(
    insets: FixedInsets(uniform: 10),
    content: ColorSurface(color: .red)
)

Here are the steps to render this to a screen of 800x600 pixels.

1. Parent Proposes Size for Child – The outer container is 800x600 so the FixedInsetContainer receives that.

2. Child Chooses its Size – The FixedInsetContainer by definition fills all available space, so it accepts 800x600 as its size.

3. Parent Places Child in Parent’s Coordinate Space – We have no additional positioning information, so the FixedInsetContainer is placed at origin (0,0) (upper left).

4. Parent Proposes Size for Child – The FixedInsetContainer subtracts 10 pixels from each side (20 width, 20 height) and proposes 780x580 to the ColorSurface.

5. Child Chooses its Size – The ColorSurface also fills all space, so it accepts 780x580.

6. Parent Places Child in Parent’s Coordinate Space - The FixedInsetContainer positions the ColorSurface at (10,10), giving it that 10 pixel inset on all sides.

Rendering

By following this process for each element, we end up with a complete description of what to draw.

let elements = [
    Element(
        type: .container, 
        rect: CGRect(x: 0, y: 0, width: 800, height: 600)
    ),
    Element(
        type: .color(.red), 
        rect: CGRect(x: 10, y: 10, width: 780, height: 580)
    )
]

Finally we can create an image or UI representing the layout!

All that work for this?

Progress

I’m happy to report that Surface has been working really well. I’ve built a few test apps, including that first Series for Mac app. The problem is, Surface isn’t well suited to respond to user input or convert to saved documents. We’ll need another tool for that. Next time!