Earlier this year, I gave a talk about more accessible maps, and while the topic was specifically about maps, the idea for the talk stemmed from experiences with other components with gesture-based interactions, like graphs.

The Problem of Gesture-Only Interactions

Android apps have several different patterns and components that usually have gesture-based interactions: maps to zoom and move around, images with pinch-to-zoom, pull-to-refresh, swipe to accept an action (MobilePay, I'm looking at you), and interactive graphs to name a few.

But gesture-based interactions don't automatically work for assistive technologies. Sure, some of them have support for, for example, a double-tap, but moving around on a graph or map is not automatically supported. Also, people with tremors in their hands might have problems performing multipoint gestures.

Who This Hurts

• Users using alternative navigation such as keyboard, Switch access, or a screen reader
• Users with tremors in their hands
• Users using the app in freezing temperatures

Why Developers (or Designers) Do This

Gestures feel natural on touchscreens, and they're also straightforward to implement. Pinch-to-zoom and drag are the obvious choices for a map or interactive graph, and they do work well if you have full motor control and two hands in use. What's easy to miss is that not everyone has the precision these gestures require, and that assistive technologies can't usually replicate them.

The Solution

There are different ways to resolve the issue, but my favorite is using on-screen buttons. It is a solution that works for most affected groups at once: keyboard users can trigger the buttons, switch access and screen reader users can find and activate them, and users with tremors can tap a button more reliably than performing a multipoint gesture. Other solutions (or, part of solutions) include adding keyboard support and adding accessibility actions.

So, let's say we have an app that uses TransformableState to handle the zooming and moving around. That part could look something like this:

// Controls zooming
var scale by remember { 
    mutableFloatStateOf(1f) 
}
// Controls position of the map
var offset by remember { 
    mutableStateOf(Offset.Zero) 
}
val state = rememberTransformableState { _, zoomChange, offsetChange, _ ->
    scale = (scale * zoomChange).coerceAtLeast(1f)

    if (scale == 1f) {
        offset = Offset.Zero
    } else {
        offset += offsetChange
    }
}

Then, we could add buttons with logic to either zoom or move the map. We have an enum with all the actions:

enum class Action {
    Up,
    Down,
    Left,
    Right,
    ZoomIn,
    ZoomOut
}

And with the enum, it's straightforward to define the handlers for each of the actions:

val moveAmount = 20f 

fun onClick(
  action: Action
) {
    when (action) {
        Action.ZoomIn -> {
            scale = minOf(
                scale * 1.1f,
                    2f
                )
            }
        Action.ZoomOut -> {
            scale = maxOf(
                scale / 1.1f,
                    1f
                )
            if (scale == 1f) {
                offset = Offset.Zero
            }
        }
        Action.Up -> {
            offset = Offset(
                x = offset.x,
                y = offset.y + moveAmount
            )
        }
        // Other directions
        ...
    }
}

This same logic applies to, for example, interactive graphs and charts. Adding buttons for all the actions that can be accomplished with gestures would be the minimum requirement, as that solves navigation for most of the groups that can't use the gestures.

Testing

• Connect an external keyboard and test zooming and moving the map using only the keyboard
• Enable Switch Access and try to zoom and move the map
• Try using the app with one hand only
• Test with a screen reader

Read More

SC 2.5.1: Pointer Gestures

WCAG success criterion about pointer gestures. Even though the name states "Web Content Accessibility Guidelines", the principles apply to mobile too.

Eeva-Jonna Panula: Beyond Gestures: Accessible (SVG) Maps for Compose Multiplatform - mDevCamp 2026

My talk from mDevCamp 2026 about improving accessibility of maps for Compose Multiplatform. The talk goes deeper into the topic of this newsletter issue, with a full example of adding button navigation for a SVG-based map.

That's a wrap for Issue #10 of Inclusive Android Apps. Have you run into gesture-only interactions in apps you've worked on or used? Hit reply and let me know!

Next month: Offline-support.

Thanks for reading!
-Eevis
eevis.codes