What are they? Why do they matter? and How can you start monitoring them in your organization?
The DORA Metrics are four metrics identified by the DevOps Research and Assessment (DORA) team as a gold standard for understanding and improving your software delivery processes.
Every company operates trying to achieve goals or KPIs in a specific period of time following frameworks like OKRs, this usually points to business goals, like achieving X amount of Monthly Recurring Revenue or have X amount of Daily Active Users. However, when trying to establish a similar methodology for a technology organization, teams struggle to define KPIs where the tech organization is completely accountable.
DORA is a great starting point to start defining, monitoring and improving KPIs that will not only measure the team capabilities that drive software delivery, but will also have an impact to business goal. And the best of all is that all of them are in complete control of the technology organization.
In this article we will explain what these four metrics are and how you can start tracking them by making minor adjustments to your software delivery and management processes.
Deployment Frequency
What it is: How often does your organization deploy code to production or releases it to end users?
Why it matters: Frequent deployments indicate that your team has the right tooling and processes to deliver new features, fixes and updates quickly, responding promptly to market demands.
Constant deployments also have an impact in quality, since smaller releases are easier to review and test leading to less probability of introducing critical bugs in production.
Lead Time for Changes
What it is: How long does it take to go from code committed to code successfully running in production?
Why it matters: This metric goes in hand with the previous one, in order to have frequent releases, you need to have the processes in place that let engineers quickly put code in production. The lead time for changes is a key indicator of agility.
Also, by having shorter lead times, you can prevent developers context switching, which can directly affect your teams productivity.
This metric can also be used as a good indicator of processes bottlenecks in a particular team or code repository.
Change Failure Rate
What it is: What percentage of changes to production result in degraded service (service impairment or outage) and require immediate remediation?
Why it matters: None of the previous two metrics matter if at the end of the day, your team is constantly introducing bugs that lead to disruptions and consequently developer distractions and lower customer satisfaction.
It is important to have the right balance between speed and quality, if one of those two suffers, you might need to revisit your software deployment processes to ensure that new releases don't introduce more problems than they solve.
Mean Time to Recovery (MTTR)
What it is: How long does it take to restore service when a service incident occurs?
Why it matters: Errors happen. It is unrealistic to think about a scenario where no errors occur when moving fast. That's why, besides trying to keep those error at a minimum rate, it is important to have the right mechanisms to quickly recover when this happens. From incident detection to mitigation. This is specially critical when SLAs are in place.
How can you start monitoring DORA metrics?
There are no standard calculations for these four metrics, they are frequently measured in different ways between each organization. However, there are some guidelines you can follow to easily track them and see whether your organization is continuously improving in their delivery practices.
The Kindor team, can help your organization quickly adopt and measure these metrics, receive automatic reports regarding your progress overtime and provide recommendations to achieve maximum productivity through these KPIs.
Deployment Frequency
Using your CI / CD tool or code versioning tool, you can start tracking when a Pull or Merge request was released to production by having a system that records every deployment.
Kindor will track your Code versioning tool and based on your specifications, like which branch to track to detect a release, it will automatically give you your progress week over week and send you notifications when anomalies are detected.
Lead time for changes
You can start tracking this metric by measuring the time it takes from the first commit to branch that you are deploying to production, to pull / merge request release. This can be directly obtained from your code versioning and your CI / CD tools.
To make this metric more useful, break this time into four sections:
Coding Time: Time spent between first commit and Pull / Merge request creation.
Pickup Time: Time spent between Pull / Merge Request creation and first Review Comment or interaction.
Review Time: Time spent between first PR / MR interaction and PR / MR merge.
Deploy Time: Time spent between PR / MR merge and production release.
By doing so, you'll be able to spot bottlenecks easier and focus on the right stage in the process.
Kindor will automatically provide you this metric and its breakdown when you connect your Code Versioning tool. In addition to this, Kindor can enhance the calculation of Coding time when you also integrate your Project Management Tool like Jira.
Kindor will automatically look for relationships between your PR / MRs and Tasks or Issues in your Project Mgmt tool and will use the Task Start Time as the initial time of your total lead time and Coding Time. This can give you a more realistic Coding time, considering that the first Commit date doesn't usually reflect the time a developer started coding the PR / MR.
Change Failure Rate (CFR) & Mean Time To Recovery (MTTR)
As mentioned before, the CFR and MTTR are as critical as the two metrics previously mentioned, however, we've seen many cases where companies have stablished processes to measure lead times and deployment frequencies but don't have an incident management process in place.
To start measuring CFR and MTTR it is important to define an incident management process before. The purpose of this article is not to explain you how to build this incident management process, however, if you are using Kindor, start tracking your CFR and MTTR can be as simple as following these 2 steps:
Define what an incident is and how you'll record it. You can start doing this using a project mgmt tool like Jira. For example, define a specific label, project or even a combination of issue type + priority to record an incident (eg. Issues of type Bug with priority Highest, or Issues with label "incident"). Kindor supports any of the combinations mentioned above.
Once the incident definition is in place, make sure that the team moves the task created through the different statuses you're interested to measure (eg. Reported / Detected, Validating, In progress, Testing, Fixed). By doing so, you'll be able to get valuable insights regarding your ability to quickly recover from an incident and whether you may have bottlenecks in the process.
Kindor is highly customizable and as long as your incidents are recorded in your project management tool and moved from a To Do to a Done state, you'll be able to track your MTTR and CFR on weekly basis even by team or project.
In Summary
DORA is the largest and longest research program of its kind and it is widely use by top tech companies around the world due to its effectiveness providing actionable insights that help you build better, faster and more reliable software.
With Kindor, you'll be able to quickly establish and adopt the required processes to measure these metrics effectively and get automated insights regarding your progress towards mores efficient and successful engineering teams.
Commentaires