Value Yonder
← On Progress
On Progress

Goodhart's Law

2 min read

Goodhart's Law and Overfitting for the Quantifiable

Link: As We May Think

Quote: "It's no longer a useful moment... A metric by itself is a goal, it's longer a real metric." - Tobi Lütke, Shopify CEO

Explanation:

Goodhart's Law states that when a measure becomes a target, it ceases to be a good measure. This is because people will optimize for the metric itself, rather than the underlying goal it was meant to represent.

Overfitting for the Quantifiable:

Lütke extends this concept to the idea of "overfitting for the quantifiable." This occurs when we focus too much on easily measurable metrics (like KPIs or OKRs) at the expense of less quantifiable but equally important aspects of a business.

Why this is a problem:

  • Distorted Priorities: Overemphasis on quantifiable metrics can lead to neglecting valuable but harder-to-measure areas.
  • Reduced Innovation: Focusing solely on metrics can stifle creativity and experimentation, as these areas are harder to quantify.
  • Ignoring the Human Element: Over-quantification can dehumanize work and overlook the importance of employee morale, customer relationships, and other qualitative factors.

Lütke's Approach:

Shopify acknowledges the value of data and metrics but prioritizes a balanced approach. They invest in understanding the "why" behind the numbers, recognizing that some of the most valuable aspects of their business are not easily quantifiable.

Key Takeaways:

  •  Metrics are tools, not goals in themselves.

  •  Don't let easily quantifiable data overshadow the importance of qualitative factors.

  •  Strive for a holistic understanding of your business, beyond just the numbers.