Software Delivery Metrics

We all can agree that measuring productivity is hard, so what metrics should we look at for software delivery performance?

The book shows that these four metrics are worth tracking and optimizing:

• Deployment Frequency: how often do we deploy?

• Lead Time: how long does it take from code committed to code successfully running in production?

• Mean Time to Restore (MTTR): how quickly can a service be restored?

• Change Fail Rate: what percentage of changes to production fail?

Deployment Frequency and Lead Time are both measuring software delivery performance tempo. Mean Time to Restore and Change Fail Rate are measures for reliability.

Accelerate's research collected over 23,000 survey responses from over 2,000 unique organizations, from startup to large enterprise. One major finding (not surprising for most, but maybe for some):

"Astonishingly, these results demonstrate that there is no trade-off between improving performance and achieving higher levels of stability and quality."

So high performance IT organizations go both faster and safer!

Why do these software delivery performance metrics matter?

"Software delivery performance predicts organizational performance and noncommercial performance."

Optimizing software delivery performance is worth doing if you want to build a high performing technology organization!

Optimizing

Why do I see the Accelerate book as a blueprint for platform teams catering for internal developers? The book shows how good practices like Lean Management, Continuous Delivery, Automation, etc affect software delivery performance which in turn affects organizational performance [1]. Medium to large companies benefit from central platform teams driving these good practices across the organization — economy of scale instead of local optimizations. If anybody asks whether your internal CI/CD tooling team is worth investing in, give the person a copy of Accelerate ;-)

The book lists a number of well-known techniques to optimize software delivery performance (Accelerate calls them "key capabilities"):

• Continuous Delivery

  • Use version control for all production artifacts

  • Automate your deployment process

  • Implement continuous integration

  • Use trunk-based development methods [2]

  • Implement test automation

  • Support test data management

  • Shift left on security

  • Implement continuous delivery (CD)

• Architecture

  • Use a loosely coupled architecture

  • Architect for empowered teams

• Product and Process

  • Gather and implement customer feedback

  • Make the flow of work visible through the value stream

  • Work in small batches

  • Foster and enable team experimentation

• Lean Management and Monitoring

  • Have a lightweight change approval process

  • Monitor across application and infrastructure to inform business decisions

  • Check system health proactively

  • Improve processes and manage work with work-in-progress (WIP) limits

  • Visualize work to monitor quality and communicate throughout the team

• Culture

  • Support a generative culture (as outlined by Westrum)

  • Encourage and support learning

  • Support and facilitate collaboration among teams

  • Provide resources and tools that make work meaningful

  • Support or embody transformational leadership

While some of the above (e.g. version control, deployment automation) are taken for granted, others definitely need investment from our side as platform teams in Zalando.

Developer Experience

What does software delivery performance have to do with developer experience? Accelerate's findings are not surprising at all:

• High performing technology organizations have better employee loyalty (eNPS)

• Employees in high-performing teams are 1.8 times more likely to recommend their team as a great place to work

• Job satisfaction predicts organizational performance

• Evidence shows that technical and Lean management practices contributed to reductions in both burnout and deployment pain

In short: Lean Management and Continuous Delivery practices improve culture, developer experience, and job satisfaction.

In 2017, we started to measure developer experience in Zalando quarterly with a simple question: "Overall, how satisfied or dissatisfied are you with your Zalando developer experience?" (1-10 scale from "Extremely Dissatisfied" to "Extremely Satisfied"):

I see Accelerate as a strategic blueprint for us to optimize software delivery performance and further strengthen the developer experience at Zalando. We use the Accelerate metrics as KPIs/OKRs and started investigating how to leverage the practices effectively for our 1200+ customers (Zalando developers). If you want to know more about our approach regarding developer experience, check out my past presentation about Developer Experience at Zalando.

What are your thoughts on Accelerate? How do you measure and optimize software delivery performance in your organization? How do you drive developer experience as an internal platform team? I want to hear from you! Please contact me on Twitter.