Support coverage models
Can you minimize disappointment with minimal outlay?
One time I sat to discuss support coverage with my boss, since we had an interesting dilemma. On one hand, we had customers (and their associated infrastructure) all over the world, and our product was deeply enmeshed into their infrastructure. Good, right? Not when things broke! The downside to being so integral to a customer’s environment is that when they had trouble with us, it meant that they couldn’t reach a lot of their critical infrastructure. When they needed support in this situation, it was urgent and stressful on both sides.
On the other hand, we were still a fairly small startup with limited resources, especially for staffing the support team. There was no way we could provide excellent support for all customers at all hours of the day or night. We had to make compromises somewhere.
My boss made an interesting point that has stayed with me: disappointment in technical support is inevitable, but it’s our job to minimize that disappointment as much as possible across our customer base. (Call this a special case of hedonic maximization if you’re feeling philosophical.) Through that lens, we can evaluate the various tradeoffs between customer expectations and support capabilities to determine how best to minimize that disappointment.
When you come down to it, there are basically four types of support coverage. In order of (swiftly) increasing resource requirements, they are:
Ad hoc
Business hours only
On-duty rotations during off-hours
Full 24/7
Each has its advantages and drawbacks, and the type of coverage your team will provide is heavily dependent on your customers’ needs as well as your own internal resources. Let’s take a look at each in order.
Ad hoc
This is the default state: no specific support hours, issues are addressed when there are resources available to address them. In a very low-volume support environment, this model can persist for quite some time until the support load becomes too great to be handled without additional structure. It is implicitly this ad hoc model when startups are handling support issues by having founders respond to emails (between fundraising, and founder sales, and engineering, and…), and becomes an increasing drain on their time until they hire one or more support-focused folks.
While this is a fine place to start, there must be a plan to evolve past this support structure to something more formalized.
Business hours only
This is the simplest, and often the first, formal coverage model for a small support team. If a customer issue comes in during specific hours, it is addressed immediately (or as soon as a resource is available). Otherwise it remains untouched until business hours resume. Depending on the customer base, these hours may be specific to one time zone (9-5 ET, Monday to Friday) or covering one or several regions (9AM ET to 6PM PT, Monday to Friday).
At bare minimum this can be covered by a single support person, though they’ll be spending their entire time on support duty.
A good support coverage model for non-critical product support, and in resource-constrained environments.
On-duty rotations for off-hours
While business-hours-only support is fine for many products, where customers are unlikely to be using the product on the weekend or an outage doesn’t cause major issues and can wait for Monday, vendors of mission-critical products don’t have that luxury. And if your customer base is global, or substantially outside your primary country of operation, you’ll need to account for that. EMEA (Europe/Middle East/Africa) starts work 5-8 hours earlier than even east coast US, and APAC (Asia/Pacific) earlier (and later) than that. And even work days vary across the world—just as examples, the work week in India is Monday to Saturday, and it’s Sunday to Thursday in Israel.
If you have customers outside your home country, guess what? You’re going to have support issues coming in on what you consider nights and weekends. If you don’t mind disappointing those customers (there’s that word again) you can set expectations that they’ll have to wait for US hours, but that’s not always an option.
The way smaller teams tend to handle this, and the way I’ve handled it on my own teams, is twofold. First, institute on-duty rotations for weekends and holidays, ensuring there’s always at least one engineer available for issues that come in while everyone else is not working. In order to compensate for the additional work, you can either pay more for these shifts, or compensate via comp time, which is how my team handled it. For every day of holiday coverage, they received a day off aside from earned PTO within two weeks of the holiday. That way, even though hours are shifted around somewhat, the overall work time for a given support engineer still remained relatively static.
Second, have someone responsible for issues that come in overnight, and some method of escalating issues to them. In lower-volume environments it’s okay to just escalate all issues directly to the person on duty overnight, but as the customer base grows, it becomes more and more important to have some way of triaging: if your overnight person is regularly getting woken up for trivial issues, they won’t be willing to take that duty shift for long. We handled it with the equivalent of an answering service that put the responsibility of escalating on the customer: if they considered it important enough to escalate, the overnight folks would page the person on duty. If not, the morning shift would respond a few hours later. (In retrospect, it is unclear whether this method of triage really did what we expected it to—some customers would happily escalate even the least important question, while others felt that the IT equivalent of a compound fracture could wait until Monday. But that is a discussion for another time.)
Permits off-hours coverage at less cost than full 24/7 support. Some customer disappointment is inevitable if product functionality is critical or most of the customer base is international.
24/7
AKA “follow the sun”, this is the coverage model of choice for organizations with mission-critical products that have a global customer base.
Unfortunately, after just a few simple calculations, it quickly becomes clear that you need a really, really large team to properly implement this. Discounting vacation/sick/holidays for now, let’s count.
8/27/23 note: these numbers are off by a factor of seven. There are 8760 hours in a year, leading to a minimal requirement of 4.2 engineers to cover those hours, assuming no overlap. See Staffing levels and support coverage, part 3 (8/28/23) for more on this. I’ll leave the original numbers below for transparency.
40 hours/week x 52 weeks/year: 2,080 hours/year/engineer.
24 x 7 x 365 = 61,320 hours per year.
That comes out to just under 30 FTE (29.48 to be more precise) to fully cover those hours.
Add in vacation/sick/etc., the fact that you can’t just slot people into whichever arbitrary hours need coverage, and the need for additional supervisory roles to effectively manage and coordinate a larger team…and those numbers start going up fast. We’ll look at this more later on, but suffice it to say for now: it takes a lot of people to do ‘true’ follow the sun coverage. But if your customer base is large enough, and the product is important enough to them, these numbers start making sense.
A high cost, high capability coverage model for mission-critical product support. Minimizes disappointment as much as is ever possible.
Now that we’ve gotten that out of the way, let’s talk about how to determine proper staffing for each of these coverage models. First … sorry, my producer is telling me we’ve run out of time. Looks like we have our topic for next week, though!
Thanks for reading Andy's Support Notes 💻💥📝!