TL;DR: We consolidated two separate analytics pipelines into a single data store. Nothing is required on your part; some numbers might look slightly different, but they're more consistent now. The rest of this post is the author nerding out about technical esoterica.
First, a brief inquiry into changelogs
Generally speaking, we like to reserve changelog entries for meaningfully observable improvements. Anything that's visible but relatively trivial — a tiny feature, an esoteric bug fix — gets relegated to the longer changelog. And anything that doesn't even clear that bar doesn't merit a public accounting at all.
This makes infrastructural changes tricky to talk about. Sometimes we cover them as behind-the-scenes posts pitched more at engineers than users, like our recent one on migrating our database to PlanetScale.
But what do you do about a massive architectural change that has an imperceptible-but-non-trivial user impact? And how can an impact be simultaneously imperceptible and non-trivial?
Let me stop dancing around the point. We have standardized and consolidated our events. Where we once had two distinct sources of truth for things like opens, clicks, and deliveries, now we have one. Everything downstream of them — automations, analytics, webhooks, and so on — works off that single source: the events log, which has been around for a while.
Until very recently, those two data sources overlapped almost identically. And if you've ever been cursed with the knowledge of maintaining two similar datasets, you know that "almost identical" is worse than "wildly divergent." So we painstakingly unshipped the dataset we internally called email events and replaced it entirely with external events, which are clearer, more constrained, and well-documented.
Stop navel-gazing! Tell me what changed!
Very specifically: the last remaining places we exposed the old source were in email-related roll-ups — looking at click rate over time in your analytics, or drilling into the analytics pane inside a sent email.
We've backfilled all of that data, and it's now keyed off external events. No data has been lost, but you might see some small changes, because the new source enforces stricter idempotency. If you see wildly divergent numbers, please write to support and let us know.
Why does this matter to me?
Honestly, the average user probably doesn't care about any of this, so thank you for being a trooper and getting this far. But if you've ever bashed your head against our analytics, you might appreciate some of the second-order effects:
- Consistency. Our numbers now add up across disparate views in the app and the API.
- Performance. It's much easier to optimize one data pipeline than to keep two of them in lockstep.
- Access. This entire dataset is already available in the UI as the events page. Later this month, we'll expose it via the API too, so you can grab whatever aggregations or pivots you want without stooping to the UI.
A plea for understanding
Analytics are really tricky, both architecturally and from a customer-experience perspective. This is one of several work streams aimed at making our analytics better in some ineffable sense — a task made harder by the fact that half of our users don't care about analytics whatsoever while the other half treat them as being of utmost importance. Harder still because "analytics" isn't really a single feature, and the data people care about varies wildly.
We want to move toward a world where we're very opinionated about what we think the majority of users will find useful, while offering programmatic access and exploratory tools for the long tail of more esoteric use cases. In that respect our philosophy mirrors how we've evolved templates: bake out the design tokens everyone cares about, while still leaving custom CSS as a backdoor.
As we do, we want to hear from you! If you have analytics-related woes, email Nick who is working on a redesign of the whole shebang.