Sunday, February 11, 2007

[muse] Improvement vs Measurement

There is a popular saying "You can't improve what you don't measure".

The statement is bit of a trap -- more often than not, it leads people to think that once they found something they can measure, they must be improving something. In reality, things are rarely that simple.

For example, companies often like to measure open bug counts, as a mean to drive improvements in software quality.

But, software quality isn't the end goal -- "Customer Success" is. In my experience, measuring bug counts alone was very harmful to customer success. In order to keep the bug counts low, the engineering team fought endlessly with QA and the customers.

I have tremendous respect/sympathy for anyone who has to come up with measurements -- It's a topic that appears deceptively simple, but in reality -- very very tricky.