There’s been no shortage of ink spilled on the topic of excellence in engineering. (Here’s a recent piece we really liked). The best articles on the subject often engage with the complex intersection of organizational design, culture, and dev practices.
Still, as an engineering manager or team lead it can be difficult to identify in practice: what are the tangible signs of an organization that is thriving? What are some of the signs of an eng team that’s not operating at its full potential?
To answer this question, we took inspiration from Ben Horowitz’s classic essay “Good Product Manager/Bad Product Manager.” It’s the iconic training manual for product management: the north star for an entire generation of PMs eager to know what excellence in their field looks like.
In the course of analyzing and reviewing data for many eng teams, we’ve identified a few concrete patterns that are associated with eng teams at different stages of maturity and different levels of effectiveness. Three key metrics provide a useful at-a-glance heuristic to differentiate between teams with different likelihoods of success – and to identify opportunities for improvement.
Despite the title of this article, I hesitate to use the words “good” or “bad” to describe eng teams, so instead I’ll defer to “high-performing” and “struggling.”
High-performing eng teams keep cycle times, from open to merge, as short as possible. (Cycle time refers to the elapsed time from start to finish of a task or project. It’s a lagging indicator of flow – a key concept for scrum teams, as described here.) The benefits of shorter cycle time include faster time-to-market and lower risk in the development process.
One of the most important drivers of cycle time is PR size. Conventional wisdom is that smaller PRs drive faster cycle time because they’re easier and faster to review. This is true, but it doesn’t tell the whole story of why big PRs tend to be disproportionately slower even relative to their size.
The graph below shows an example of the cycle time for a high-performing team. Cycle time is plotted against PR size here to show how much of a team’s cycle team on individual tasks is driven by PR size (since larger PRs take longer to code, review, and merge). Many of this team’s PRs have a cycle time of < 24 hours, supported by small PR size. At most, larger PRs have a cycle time within a 4-day range.
Struggling eng teams tend to have longer cycle times, often driven by large PRs (which may signal opportunities to improve the planning and estimation process). As described above, large PRs result in disproportionate “drag” on the rest of the review cycle. The chart below shows a typical pattern for a team with unwieldy, improperly-scoped PRs: roughly 50% of PRs having a cycle time of 10 days or greater.
A related pattern of struggling eng teams is large PRs that end up getting merged too quickly relative to their size – often indicating hasty “rubber stamping” or an inadequate review and control process, as shown in the graph below.
High-performing eng teams make continuous strides to reduce cycle time. As described previously, minimizing PR size is a key lever because it results in a virtuous cycle: we observe from the teams we work with that when a project is broken down into smaller tasks, team members a) are generally happier to review, because they know it will be a reasonably small ask of their time; and b) are more able to find time (between tasks, during lunch) to dedicate 20 minutes to review. The graph below shows how a team that has progressively reduced PR size has reaped disproportionate benefits in the other review stages.
Struggling eng teams tend to see PR size trending in the wrong direction over time, as a project progresses. They define tasks with progressively less rigor and precision, which leads to large PRs – in turn bloating the entire review process (larger tasks -> no one wants to review -> spend longer waiting for review -> review itself takes longer -> spend longer waiting to be merged).
High-performing eng teams keep wasted work – the hours associated with abandoned PRs – at a low and constant level, ideally no more than a few hours per month. (Note that it should not be a goal for eng teams to have zero wasted work, as this typically signals a team that isn’t taking risks on new approaches or moving quickly to act on emerging opportunities.)
Struggling eng teams demonstrate wasted work at a high level (below top, showing ~80 hours of wasted work per month) or increasing level (below bottom, showing wasted time on the rise). Both scenarios point to a) a need for stronger cross-functional alignment between engineering and PM on the goals and definition of success for a given product initiative and or b) more proactive engineering management and coaching on the scoping and execution of key tasks.
These three metrics provide a diagnostic snapshot into the operation of an engineering team at a given point in time.
But whether a team is high-performing or struggling, the ultimate goal can and should always be continuous improvement. Tracking and monitoring changes in performance over time – whatever baseline the team is starting from – is the key to identifying focus areas and unlocking the team’s full potential.
Get instant visibility and the actionable insights you need to maximize your dev team’s potential!
Book demo
