We Are Measuring Developer Productivity Wrong

Table of Contents

• What business want?

• Business Sees Development Teams as Manufacturing U …

• Problems with Common Metrics

• How About Talking to People?

• This Isn't the End

Measuring productivity is challenging in any field, but it's especially difficult for developers due to the nature of their work. Developers face different tasks every day, making it hard to apply a consistent productivity measure. So, how can we effectively gauge developer productivity under these conditions?

What business want?

Let's start by understanding what businesses expect from developers. If we grasp this, it becomes easier to define how to measure developers in a meaningful way.

Why do businesses want to measure developer productivity? Initially, when a business is trying to find its product-market fit, everything is about speed. Developers build features quickly, each new feature contributes directly to growth, and everyone is happy. The flywheel spins, and the business starts hiring to keep it going.

But eventually, the growth curve flattens. The team becomes larger, tech debt piles up, and development slows down. In the eyes of the business, the team appears less "productive." Management wants to optimize costs and increase profits, ideally getting back to the days of blazing-fast feature development.

The business senses that the team isn’t productive enough, but they don't understand why or how to improve things. They turn to the VP of Engineering or CTO, asking for metrics to prove that the team is working well and delivering value. The pressure is on for leadership to find some numbers that can "prove" the team's value and help them make decisions about performance. Ultimately, the business wants:

• Faster delivery of features and projects to drive profit.

• To identify underperformers for potential firing and top performers for rewards or promotions.

• Metrics that strongly correlate with company profit so they can effectively manage the team.

Business Sees Development Teams as Manufacturing Units

For tech companies, developers are often viewed in a similar light to factory workers in a car manufacturing plant. Every feature is seen as a competitive advantage, directly linked to profit. If the process of building new features can be sped up, it’s like opening a tap that pours out money.

Manufacturing companies optimize by measuring every step, creating a clear map of how work flows, and constantly fine-tuning to produce more cars faster. The temptation is to apply the same thinking to software development.

You might say, "But software development isn’t like manufacturing; each feature is new and unique." And you’re right. Every feature is different, and we can’t simply break it into tiny pieces and speed up each part independently. Development is iterative—you make one change, then realize you need to adjust three more things, and sometimes you have to rethink the initial change entirely.

On the other hand, there are consistent processes we follow. We always do code reviews, sometimes testing, and other types of reviews. There are upstream processes like ideation, design, estimation, and planning that we repeat every time. So, while every feature is unique, the process around it isn’t. What if we focused on measuring these processes?

Problems with Common Metrics

Let’s look at some commonly used metrics: lead time for changes, deployment frequency, story points per developer, issue cycle time, throughput. Most of us have used or at least heard of these metrics, but why not just stick to the popular ones?

1. Limited Scope: Many of these metrics focus solely on the development process and ignore other crucial steps. For example, DORA metrics are about deployment frequency, lead time for changes, change failure rate, and mean time to recovery. But they often miss QA, product management, or other stages.

2. Easy to Manipulate: These metrics can be gamed. If you set a goal to double the story points completed in six months, developers might simply estimate tasks higher or split them into smaller tasks to inflate the numbers. Suddenly, your team’s story points have doubled, but business outcomes remain the same.

3. Lack of Diagnostic Power: Metrics like throughput can tell you there's a problem but not where it is. Is the throughput low because developers are slow, QA is backlogged, or features are waiting on approvals?

4. Team-Specific Variability: Different teams have different processes. Metrics like deployment frequency might be irrelevant if your customers don't need frequent deployments. You can't just tell a team to change their process to hit a metric without considering their unique context.

5. Weak Connection to Business Value: Many popular metrics don't have a direct link to business outcomes. The focus is often on technical performance, which doesn’t always translate into tangible profit.

How About Talking to People?

To be honest, the cheapest and simplest way to improve team processes is through talking—1-on-1s, surveys, and retrospectives are great tools if you want to improve. But there are challenges here as well:

1. Bias: People are inherently biased. We tend to look for information that supports our existing beliefs (confirmation bias) and assume that others share our beliefs and behaviors more than they actually do (false consensus effect). This makes it hard to get objective insights.

2. Limited Perspective: People can only report what they see. Developers might say code reviews take too long, while QA might blame developers for slow progress. But the real bottleneck might be something else entirely—like a cumbersome release process. When I implemented metrics, I found that 30% of our cycle time was just waiting for releases—something that was easier to automate than fixing code reviews, as the team pointed out.

3. Self-Protection: People tend to protect themselves and deflect blame. Nobody wants to admit they’re the reason a project is delayed by two months. It’s human nature to avoid feeling guilty or being the "bad guy."

This Isn't the End

There has to be a better way to measure and improve productivity—one that acknowledges both the business's need for metrics and the unique nature of software development.

In my next article, I’ll explore how to choose metrics that matter, how to implement them effectively, and how to use them to get real results for both the team and the business.