Categorizing support issues
And as for the bucket, Pawtucket
A few months ago I talked about the different types of issues that typically come into the support inbox. This week I’d like to talk about a related, but distinct, topic: how do you get a handle on the kinds of support issues your specific team is seeing? What are the commonalities? Are there some overarching themes? Are 90% of your tickets about three or four recurring topics, or are things more spread out? All of this is important to understand not just for today, but for planning for tomorrow.
What happens if you don’t have any strong idea of the kinds of tickets your team is handling every day? Consider:
How can you defend prioritizing feature X or bugfix Y if you can’t point to specific customer issues caused by the lack of X or Y?
How can you know what specialized skills your team needs in your next hire?
How can you decide what kinds of support issues to focus on when training new engineers on your team?
How can you tell whether your product documentation is sufficient or lacking in specific areas?
Without a strong understanding of not only what your engineers are solving for your customers, but how much time they’re spending on it, your job is made much harder.
Analyzing tickets
When you’re working in a very new startup, and your product hasn’t been put in front of many customers yet, you’ll have to make some assumptions about the kinds of problems customers might have. A couple of categories are pretty much guaranteed, as they are evergreen across all software:
Bugs: always a safe bet, particularly with very early software.
Feature requests: ditto. As you gain new customers, they’ll come in with their own expectations of what your product will be and their own use cases your product designers may not have anticipated.
Beyond that, however, you’re going to have to use your own judgment and what you may have seen before with related products. For example, if your company is selling a networking-related product that runs on Linux, you can probably anticipate issues around:
Networking architecture
Linux command line
Supported distributions
Encryption options
Hardware/resource requirements
How can I run this on Windows/why doesn’t this run on Windows?
In each of these cases, you can prepare your team—or just prepare yourself, if you’re still running a solo show—on each of those topics and where customers might run into issues. You might put together some documentation around suggested architecture, or minimum system requirements, to head off some of the queries entirely, or at least make them trivial to address.
But once you start getting actual customer issues, those assumptions you made are going to bang against the reality of where folks are actually having problems. Once you’ve got a month or two of real customer issues under your belt, it’s time to take a step back and start figuring out your actual support issue demographics. There are three major types of information to gather, in increasing order of importance:
The actual categories of support issues you’re encountering. This will probably overlap quite a bit with your earlier assumptions, but there will likely be a few surprises as well!
A rough breakdown of volume per category: how many tickets are networking-related? How many are around Linux troubleshooting?
A breakdown of time spent per category. Maybe you have 50% of your tickets around architectural issues, but you’re only spending 10% of your time on them. Meanwhile you’re only getting a handful of tickets around bugs, but your team is spending 70% of their time addressing those issues.
I believe that last is the most important because it really gets at the core of your concerns as a support leader: what is my team spending their time on? Any category that is taking up a lot of your team’s time, particularly compared to the actual number of issues customers are reporting on that topic, is a bright flashing signal for you to investigate further. That being said, the other two bullet points are key as well. If your team is fielding lots of questions one one particular topic, even if it’s not taking them long to respond, that’s a sign that something is not as clear as it should be. Maybe documentation needs to be improved, maybe the product itself needs to be made more usable, maybe there’s a nagging UI bug that needs to finally be addressed.
The overall categorization of your incoming tickets is important both as a grounding context for the ‘most common tickets’ and ‘most time-consuming tickets’ bucketing, and to give you some strong hints as to the skillsets your support team needs to have. If you’re fielding a mix of networking and Linux tickets, you’ll be looking for a different technical background in your hires than if you’re spending 90% on Windows desktop issues. Sadly, while ‘ticket count per category’ and ‘time spent per category’ are comparatively strightfoward to calculate, figuring out the correct support issue categories and how to effectively categorize your incoming issues is going to be a lot more complicated.
Some notes on categorization
I think I should probably spend some more time on this in a separate post, but I’d be remiss not to talk a little about how this categorization might actually happen. In the early days of a support team’s existence, it’s not overwhelming to review every ticket that comes in and start creating provisional categories. If your ticketing system has tags, that might be a good early use for them—tag your tickets with something plausible, then after a few weeks do a review where you look at each ticket, and each category you’ve come up with so far, and decide what makes sense in your situation. If you’re handling lots of networking-related tickets, for instance, it may make sense to subdivide your own categories into ‘networking - Linux’, ‘networking - Mac’, ‘networking - Windows’. But if they’re only a small part of your overall ticket volume, it probably will be more effective to put them all under a single ‘networking’ category.
The overall number of categories you end up with is important too. Too few and you will have a hard time drawing useful conclusions, but too many categories and you risk many of them being so specialized that they are rarely used. In my experience, 10-15 overall categories is what you probably want to aim for. Don’t forget to have an ‘other’ or ‘catch-all’ category, too, because not every ticket is going to neatly fit into your carefully designed buckets.
The longer you wait to go through this process of determining categories, the tougher it’s going to be, especially when there are more and more people on the team who need to agree on which categories make sense. The larger your ticket volume, too, the trickier it is to review every ticket to determine categories—you’ll have to resort more and more to a sampling-based approach. But if you haven’t started yet, there’s no time like the present. The benefits you’ll gain from this exercise are worth the initial tedium.
As you can probably imagine, determining appropriate ticket categorizations is not going to be a one-and-done project. The mix of tickets you’re getting in Year 1 may be very different from the mix a year or even a few months later. Don’t assume that since you’ve optimized your team to handle today’s Top 10 issues that you’re good for the foreseeable future. Periodic review of ticket tags is a good start, but it’s important to periodically (maybe quarterly) sit down with team leadership to update your ticket statistics and your top issues. Pay close attention to that ‘other’ category, because newer trends in support issues will often show up there first. If you notice the number of tickets in that category growing, that’s a sign to look more closely and see if you need to create a new category of issues.
Having a clear understanding of what issues come up the most, and what is taking the most of your team’s ticket handling time, is a critical component for strategic and capacity planning. If you don’t take the time to do it now, you’re missing out on information that’s critical to your team’s success.
Thanks for reading Andy's Support Notes 💻💥📝!