Skip to content
BinaryScaler

Data

The metric that disagreed with itself

Two dashboards, one metric, two numbers. The cause is almost never the pipeline — and knowing that changes where you start looking.

Written by
Dana Whitfield · Principal Engineer
Published
27 March 2026
Reading time
7 minutes

A client once showed us two dashboards side by side. Both said 'Active Customers'. One said 84,000. The other said 61,000. Both had been used in board papers, in the same quarter, by different departments.

The instinct is to look for a broken pipeline. There was not one. Both numbers were computed correctly from correct data. They simply answered different questions, and neither dashboard said which.

Definitions drift silently

One team had defined active as any login in ninety days. The other used a transaction in thirty. Both definitions were reasonable when written. Neither was written down anywhere the other team would look.

  • The definition lived in SQL, in a dashboard, in a tool one team used
  • The person who wrote it had explained it verbally, once, in 2023
  • Nothing broke when it diverged, so nothing signalled the divergence

A semantic layer is a definition with an address

The fix is structural: define the metric once, in version control, and make every consumer read that definition rather than reimplementing it. If two definitions are genuinely needed — and sometimes they are — name them separately so nobody can quote the wrong one by accident.

metrics:
  - name: active_customers_30d_txn
    label: Active customers (30d, transacting)
    owner: finance
    definition: count(distinct customer_id) where last_txn_at > now() - 30 days

Start with the metric people argue about

Every organisation has one. Trace it from dashboard back to source, write down what it actually computes, and the shape of the wider problem becomes visible within a day. That is a better opening move than a governance programme, and it produces something usable immediately.

  • data
  • analytics
  • governance

Author

Dana Whitfield

Principal Engineer

Fifteen years in payments and platform engineering. Writes about the operational side of delivery.

Meet the team

Want this applied to your system?

We will take a look at what you are running and tell you which of the above is worth doing first.