Winning 2023 - practical tips for using the SPACE framework

DORA metrics* have become very popular over the years, and as a data lover myself, I’m very happy with that! However, many forget that DORA metrics don’t guide your work toward success; they measure it.

If you have your DORA metrics in front of you, you are an all-star, but you still need to find your way of doing things to realize your team’s full potential and stick to it (which is usually the hardest part). This is especially important during economic slowdowns—but also tougher.

Dr Nicole Forsgren, one of the 3 contributors of the book Accelerate,* presented the SPACE framework, a guide for engineering leaders to look at their team productivity in a holistic way, considering 5 dimensions.

Many engineering teams I’ve worked with over the years followed some parts of it, intentionally or intuitively, and it was never a bad decision.

The following is my take on SPACE in challenging times when business goals are aggressive and resources are limited.

Satisfaction & well-being

Some of your developers have friends and family that were laid off. Now more than ever, take care of their well-being. Make sure they feel fulfilled with their work, motivated teams always deliver more!

Here are a few simple methods you can implement very easily to get a significant impact:

• Surveys
  Use surveys, preferably anonymous, to ask the hard questions and identify your team's sentiment. Make sure to ask your engineers if they feel burnt out and how they feel about their efficacy and self-accomplishment. Understand where they would like to see themselves within the engineering team in the future and act accordingly.
• Psychological safety
  Make sure your engineers feel safe to express their concerns, needs, and thoughts. For example, when done well, the retrospective meeting is also a safe place for engineers to share their opinions, satisfaction, and happiness with the teams’ work. When psychological safety is in place, you’ve already increased their satisfaction.
  
  💡 Tip
  You can and should infuse surveys and polls into your retrospective process. You will be amazed at how many insights you will get from asking simple questions like “how happy were you with the Sprint” every Sprint. Your goal will be to see the happiness rate increase, despite the slowdown and the workload.‍
• Concurrent work
  Use data to detect who is most probably dissatisfied, who is perhaps more burned out than others, etc. A simple data point to follow is the number of PRs the team worked on, a spike that might indicate they are working overtime to deliver their tasks. In that case, consider a recharge day—who doesn’t want to re-energize? ;)
  

Performance

The best engineers you can ask for are soul players, influencers who mentor the team and invest in reviews to make the team shine and the product you deliver, the best. In an economic slowdown, these key performers are a crucial ingredient to guaranteeing your success. To turn engineers into true soul players, make them aware of the outcome, results, and influence of their work on your customers; they in turn will care more about what they develop.

Here are a few popular ways to do so:

• Customer satisfaction surveys (CSAT) and NPS
  If you are running CSATs or NPS, share the results with your engineers. Keep in mind that many of them are not familiar with what these surveys are about and the meaning of benchmarks. Make sure you present all the additional info they need to understand their contribution to the business.
  
  💡 Tip
  Most of the survey tools support integration into communication apps such as Slack, Teams, Discord, and more. Use it! Make customer feedback accessible to all your engineers (or anyone in your company for that matter).
  If you don’t run customer surveys (and frankly, even if you do) you can always show usage of the features you track via your product analytics tool. Any engineer will be happy to know that customers are using the feature he just shipped!

• Critical bug frequency & MTTR
  If you are not measuring your code quality and the time it takes to fix critical issues in production, you should start, now! Maintaining high quality is standard today for providing the best customer experience. However, making sure your engineers are aware of the quality of their work is just as important. Show the MTTR and bug frequency to your engineers so they can understand the end-user experience and the final result of their coding—yes, bugs are also part of the results. This will increase their sense of urgency and production awareness.
  
  💡 Tip
  In order to have a metric that remains relevant as your team or its throughput grows, you can measure the ratio between the work done and new bugs. We call that defect density (e.g., new bugs/merged pull requests or new bugs/new lines of code).

Activity

If you had to let people go, if you stopped your hiring plans, or if you just have to deliver 150% more than usual to keep up with the business during this economic climate, then looking at the number of actions or outputs of your engineering team is crucial to confirm that you are up to speed. 

• Deployment frequency
  Check how often you ship features into production—it’s a good indicator of your team’s velocity. Although you should aim to be in the elite group that deploys on demand, during stressful times you should first try to maintain the existing frequency of your deployments and avoid deterioration. Make sure you have your deployment frequency chart in front of you and then set small achievable goals to increase your deployment frequency. 
  
  💡 Tip
  Setting achievable goals to improve deployment frequency based on internal benchmarks is far more practical then comparing yourself against the general industry benchmarks. Having said that, it’s still good to know where you stand! Here are the deployment frequency benchmarks according to the DORA research and what we saw works with thousands of teams:
  - Elite: We mostly see 1-2 deploys per day in the best companies.
  - High: We mostly see 1 deployment per week and up to once every two weeks. 
  - Medium: Less than one deployment per month goes deep into the slow territory.
  

• Throughput
  Another metric you should be following, especially during an economic slowdown, is your throughput. Throughput represents the rate of work done by your engineering group and so it quantifies how much you deliver. This can be measured by the number of merged PRs overtime, completed tasks and even story points delivered.
  When business goals are aggressive and competitors are breathing down your neck, make sure your throughput is steadily trending positively. Yes, even if you have to let people go, you are expected to deliver
  
  💡 Tip
  ‍Take your throughput chart with a grain of salt. The idea is to look at the trend, identifying if you have a concerning decrease or an optimistic increase. If you have the data in front of you, check which of your teams had a decrease in throughput, try to understand why and how to prevent that—helping the “weaker” links is important for driving your engineering group as a whole towards success!

Communication & Collaboration

Your engineering talent potential can be top-notch, but the way they work together and collaborate will determine if you would realize that potential or not. Communication and collaboration do not stop at happy hours (even though we love them). Here are some practical measurements you can look at to improve team’s collaboration, and in turn, boost productivity and delivery.

• Team influencers
  Identify the engineers that are top influencers. These team players usually mentor other team members, review PRs carefully with plenty of valuable feedback, and are the first ones to help with the heavy lifting doing critical bugs and high-priority tasks. Generally speaking, they drive the team forward. As an engineering manager, make sure they know you acknowledge them and show your appreciation.
  
  💡 Tip
  A few ways to detect team influencers is by looking at the top reviewers in the organization (for example, look for the engineers in the top 10% percentile of reviewed PRs). You can also look for engineers who review the PRs of many other engineers or detect the heavy lifters, the ones who take on most of the high-priority issues.
  ‍

• PR Cycle time and cycle time breakdown
  PR cycle time represents the time between the first commit and until the pull request is merged. This metric shows how quickly you ship software to your customers; lower PR cycle times usually result in higher output and efficient teams. PR cycle time includes a few phases: coding, PR waiting for review, PR in review, and PR pending merge. The tighter your team collaboration and communication is, the shorter the cycle time will be because reviews will be picked up quicker and will be handled efficiently.
  
  💡 Tip
  Don't just look at the PR cycle time; look at its breakdown, and propose the team to use different alerts and reminders to improve a certain part. For example, use the GitHub plugin for communication tools to send the team a reminder about PRs waiting for review every morning on Slack, Teams or any other communication channel you use.

• Review cycles
  Review cycle represents the number of times a PR goes back and forth between the author and the reviewer. Whenever a PR is passed back and forth, the submitters and reviewers involved need to context switch and invest more of their time on the same piece of code, instead of working on new features. When your engineering team is required to deliver quickly with less coding hands, the last thing you want is PR cycles to be a bottleneck. This is demotivating for the engineers and you need them focused, rather than scattered around multiple PRs.
  But what can you learn from review cycles about your team’s collaboration? Well, if the review cycles are high, this might indicate the team is not aligned on how a review should be done, or that there’s confusion around how features should be implemented. 
  
  💡 Tip
  Check out who is typically reviewing PRs for certain developers, this can teach you who in your organization is collaborating more with one another. Also, if some PR authors tend to have longer review cycles than others, they might be struggling a bit, so keep an eye out!

Efficiency and flow 

Understanding how well developers can process and deliver without interruptions or delays is key to keeping your team productive, especially when they have to deliver more with less resources and time. Optimizing engineering processes and keeping work efficient can be easier than you think! For example, you can’t have bad review processes slow you down. Even a simple assignment rule or reminder on open PRs can shorten your cycle time by 50%! Sometimes a quick refinement in your deployment process can help your engineers ship much faster.

💡 Tip
Give your team the full ownership of improving their own processes. Let them come up with ideas they feel comfortable with and the change will come—and stick—very quickly!

Here are a few metrics you can follow and use to optimize your process:

• Unplanned work and bugs
  Unplanned work is part of every engineering team’s day to day. Especially during critical times, your business team will do its utmost to keep customers onboard or expanding with new features they always wished for. They might even push for a quick pivot to make the sales processes more effective to adjust to the economic slowdown. This typically brings even more unplanned work. However, you still can and should keep your engineers from being distracted and their planned work from being constantly interrupted. One way of doing so is to monitor your unplanned work along with more indicators. For example, if an increase in unplanned work is coupled with a decrease in output, or with a concerning increase in cycle time, this means the unplanned work is affecting your delivery in a serious way. work with your product and business teams to keep disturbance only when it’s really necessary.
  ‍
• PR Cycle time and cycle time breakdown (yes, again)
  We already touched on PR cycle time, its breakdown, and the importance of each part. But we could not write about flow and efficiency without stressing again how important it is to not stall unmerged code for too long. If you do not pay attention to your cycle time in general, particularly the time it takes to pick up a review or merge a PR, you will suffer from lower quality, more conflicts upon merging, developer context switches and loads of frustration.
  
  💡 Tip
  Cycle time is the one metric that is the easiest to improve. When broken down properly, you get actionable ideas as to what to improve—for example, create processes and alerts to encourage reviews, streamline PR merge, break down tasks into smaller PRs and deliverables and more. Having said that, when engineering productivity is critical for the success of the company, encourage each of your developers to have a sense of ownership of their individual PR cycle time. This in turn will result in creating smaller manageable PRs, in developers pushing their own PRs for review and merge and more.
  

Most importantly, don’t freak out! You do not have to implement all 5 dimensions of SPACE in order to boost your team's productivity. We get it, it’s too much to commit to. 

Choose 3 dimensions to focus on. Your team will appreciate a leader who acts in tough times and fosters a culture that sets them up for success.

‍

‍

*What is DORA?

The DevOps Research Program (DORA) team at Google publishes a series of annual reports to benchmark processes and capabilities that help teams achieve high performance in software delivery. In their 2018 book Accelerate, the DORA team identified a set of metrics which they claim indicates software teams’ performance. These metrics have come to be known as DORA metrics.