Making thoughtful changes to engineering teams
Manager, don’t kill my vibe: how to change team processes without drowning in metrics.
It's difficult enough to change social systems. Changing how teams work is hard because software development is an inherently human, social challenge. Engineering managers are often asked to go a step further and measure the effectiveness of the changes they made. It's reasonable to ask leaders to consider and demonstrate the effectiveness of their changes. But it's also easy to fall into the false promise of scientific management and think that knowledge work can be distilled into numbers and percentages.
In short: for changes in human/social systems, use subjective data collection to discover the objective measures that might work.
1. Blatantly un-scientific management
Software development is highly social. People matter. In social activities, vibes matter. They're not the only thing, and you can't wave off proposed changes or ideas because they have a "threatening aura." But you also can't run a successful software project with people who dislike how they're working and feel like their days are filled with tedium and wasted effort.
The trick is finding measures that support the team's energy/vibes rather than inhibit them. Rather than impose tedium on teams, I pair changes to how the team works together (process, team composition, pace) with subjective measures. I ask simple questions: "How is this change going?" and "Are we closer to solving the issue we faced?"
In the process of discussing that, I'll either get the impression that we're on the right track or that we need to step back. If a step back is the right move, I consider the difference between what I'd hoped would happen and what actually happened, including all the unintended consequences of the change. Then it's back to the drawing board to make another change. Every so often, the next move is rolling back my idea and sticking with the status quo that we knew.
If the change seems to be working, then I start thinking about objective measures. But I'm careful here—I don't want to create tedium and overhead for my team. People do their best work when they're focused on building things that matter to users, not feeding data to dashboards. So when I do measure, I look for numbers that already exist: PRs shipped, escalations resolved, new customers integrated.
2. How I introduce change to teams
In a nutshell:
- Tell the team about the problem I'm observing and hoping to solve. Ideally, we've discussed the concern before, as a team and individually. And I've talked to folks about the shape or intent of solutions. See also, Don't Be Spooky.
- Describe the change I'm making to how the team works, collaborates, or coordinates. Again, the team needs to buy into both the problem and the change. I should have already started socializing ideas about what a practical solution to this problem looks like and how we might do it. At this point, it's a proposal, not a mandate.
- Describe the outcome I'm going for. I think of this as a fitness function for the proceeding steps. I need to connect the dots from the problem through proposed changes to a newer, better world. If writing the outcome is difficult or seems specious, I need to iterate on the problem statement and proposed changes.
- Share the problem, solution, and intent in a document. This is the first opportunity for the team to give constructive but critical feedback: "how does that change solve that problem?" Ideally, I share it as a proposal first, get feedback from the team asynchronously and in working sessions, then we talk about it as a team to decide upon action.
- In the course of normal work, ask individuals and the team how the change is going. This happens on many channels. This includes individual 1:1 chats, assuming there's time beyond what the direct report wants to discuss. I use a periodic and systematic survey tool to poll my project teams (I happen to have built one of these, so I was dogfooding here). Add it to your team retros, if the team is okay with that.
3. Caveats
Don't apply this where Goodhart's Law will invalidate your measures (when a measure becomes a target, it ceases to be a good measure). Subjective measures are handy here, as they're inherently not "scientific". That said, "3 out of 4 developers liked this change" or "2 out of 4 developers feel this is slowing them down" are useful signals. Four subjective responses are not a data point, but four sentiments extracted from those responses are actionable data.
Anything worth gaming, engineers will game. You should probably think twice before using this for performance management and compensation schemes.
4. TL;DRs
Don't assume you can't make changes because they aren't measurable.
Don't create tedium for the sake of measurement, e.g., TPS reports.
Iterate on your team's process, organization, collaboration, and coordination. Avoid big bang changes like re-orgs.
Ask people how it's going and what they think. Reflect on how that squares with your intent. Let the vibes lead you to better vibes. 🙃