The Engineering Metrics Benchmarks data was collected from 930 dev teams in the USA, Europe and Israel.
Engineering metrics are quantitative measures of various aspects of software development and engineering processes. They provide a way to track progress, identify areas of improvement, and make data-driven decisions.
Pull request cycle time is a “super engineering metric״. Measuring and analyzing it allows you to understand many hidden patterns and behaviors in your team. It can show the efficiency of your team and the quality of teamwork and collaboration in your team, and it directly impacts the quality and velocity of your team. Long cycle time usually means a bottleneck exists somewhere in the process. In order to look into it and understand where the bottleneck is and how to fix it, you need to dive into the different phases of the cycle time, as described below.
Short coding time correlates to low WIP, small PR size and clear requirements. In case you find that your bottleneck is around this stage, you should look into:
Short waiting time represents strong teamwork and a healthy review process. In case you find that your bottleneck is around this stage, you should look into:
Similar to coding time, the review time depends on the PR size and complexity and the quality and clarity of the requirements. In case you find that your bottleneck is around this stage, you should look into:
Your pending merge time can be affected by several intra and inter team factors. In case you find that your bottleneck is around this stage, you should look into:
Deploy time refers to the time between the merging of a branch and its release. It’s a great way to measure your employment process and automation. Low deploy time correlates to high deployment frequency.
Similar to merge time, deploy time is also affected by the complexity and size of the changes. Smaller and more frequent changes will result in shorter deploy time.
Deployment frequency is a measure of how often code is deployed. A stable and healthy continuous delivery pipeline means faster deployment frequency. The best way to improve deployment frequency is to automate the build and deploy process and implement CI and CD.
Deploy frequency is also a result of the previous metrics—cycle time and deploy time. The shorter they are the higher the deployment frequency is, therefore once again, the smaller and more frequent the changes the more frequent deploys will be.
Shows the average pull request size, i.e., the amount of changes in lines of code, includes additions and deletions. PR size has a tremendous impact on almost any metric in your development process, from coding time to deployment frequency.
Reducing and keeping small tasks and small PRs is the first step to a healthy and stable process.
Try to set and track an objective like “90% of new PRs below 250 lines of code.” Even if you focus just on this, it will bubble up and improve the rest of the process.
Mean time to resolve is the amount of time spent from a bug or incident being opened until it is ultimately resolved. Probably one of the most important quality metrics, MTTR directly impacts your SLA and customer experience. Time to resolve should be measured differently per priority, severity, type and even industry. High priority production incidents should not be treated the same as low priority front-end bugs. Each company should define its goals for MTTR according to its SLA. Start from defining a goal for highest priority bugs MTTR; this will get the team aligned and set the right expectations and priorities.
Defect density is the ratio between the amount of bugs opened vs the amount of pull requests merged. The most common metric for engineering teams is bug count. It’s easy to extract and easy to understand, but it is very hard to measure it over time, when team size, team velocity and product complexity change. Bugs count should be normalized to the amount of work done. Defect density solves this by showing the ratio between the amount of work done to new bugs. Setting a goal of 25% ratio (new bugs/merged PRs) usually represents a good benchmark. One way for your team to improve it is to reduce the PR size and increase the number of PRs, thus reducing defect density. It might look like cheating but we encourage this behavior. Smaller PRs will lead to higher efficiency and higher quality.
Traceability shows the amount of pull requests linked to a ticket in the project management system. Although traceability is not a common metric, it has become very important in recent years due to security and compliance requirements. It is also important to make sure that any work done, every line of code, has a business requirement. The best way to improve traceability is to implement a process of adding the ticket ID to the branch name. Some systems, like Acumen, use AI to link your code and tickets and improve your traceability levels.
Tell us a bit about yourself, and we'll set up a time to show you how dev teams boost productivity and predictability with Acumen.