Skip to main content

Metric trust, drift and duplicates

Most companies do not have a data problem. They have a trust problem. Two dashboards show different revenue, a meeting stalls while people argue about whose number is right, and the decision slips to next week. Flynt is built to stop that, by keeping the definitions behind your numbers honest as your team grows.

This page explains the safeguards, how drift and duplicate definitions are detected, exactly how the matching works (including the common question, "how similar do two definitions have to be?"), and how to use it day to day.

The three safeguards​

1. Safe by construction. Every query a person or an AI writes in Flynt is read-only and runs under a cost ceiling. A query can never change your data, and it cannot run up a surprise warehouse bill. This is the floor everything else stands on: people can explore freely without anyone worrying about damage.

2. Drift warnings -- the same name, a different definition. Once a metric is certified, Flynt remembers its definition. If someone later builds a card with the same name but a different definition, Flynt flags it at save time. The certified definition is the source of truth; new work that disagrees with it is surfaced, not silently accepted.

3. Duplicate definitions -- the same number under different names. The quieter, more damaging problem. Five people each build "Active Customers", "Active Accounts", "Live Users", "Engaged Customers" and "Current Customers", and four of them are the same query. Flynt detects this and lets you pick one canonical definition and retire the rest.

On top of these, every metric in the dictionary tells you where it comes from (the tables and columns it reads), how fresh it is (when it last ran), and how used it is (views and distinct viewers in the last 30 days). A number you can trace is a number you can trust.

How certification sets the baseline​

Nothing enters the dictionary until it is certified. When you certify a direct-SQL card or a dashboard, Flynt records its SQL as the canonical definition for that metric name. Uncertified experiments in the playpen never pollute the dictionary. In other words: you decide what "good" looks like by certifying it, and everything else is measured against that.

How matching actually works (and how similar is "similar")​

This is the part worth understanding, because it is deliberately simple and predictable.

Flynt compares definitions by canonical form, not by fuzzy similarity. Before comparing, it normalises each query so that cosmetic differences disappear: spacing and line breaks, keyword case (select vs SELECT), comments, and identifier casing. Two definitions are treated as the same when, after that normalisation, they are identical.

There is no similarity percentage and no confidence threshold. A pair of definitions either reduces to the same canonical form or it does not. This is a choice, not a limitation we hide:

  • No false alarms. You are never shown a "these are 80% alike, maybe reconcile?" guess. If Flynt says two metrics are the same, they are literally the same query written differently.
  • Explainable. Every flag can be justified to the person whose work was flagged, which matters when you are asking them to change it.

What this does catch:

-- Flagged as duplicates: identical once formatting and case are normalised
SELECT SUM(amount) AS total FROM sales
select   sum(amount)  as total   from sales   -- same query, just typed differently

What this does not catch (today):

-- NOT flagged: genuinely different queries that happen to produce the same number
SELECT COUNT(*) FROM orders WHERE status = 'paid'
SELECT COUNT(*) FROM orders WHERE status NOT IN ('cancelled','pending')

Recognising that two different queries compute the same result requires semantic analysis (reasoning about what the SQL means, not just how it is written). That is a deliberate next step on our roadmap, where AI assists the dictionary. Today, Flynt is exact and honest about its scope: it keeps the same definition from fragmenting across names, and it flags when a name's definition changes.

Using it day to day​

  1. Certify your good work. Certifying a card or dashboard registers its definition as canonical. This is how you tell Flynt what to protect.
  2. Watch for drift warnings. When you save a card whose name matches a certified metric but whose SQL differs, you will see a warning. It does not block you; it tells you that you are diverging from the agreed definition so you can decide deliberately.
  3. Reconcile duplicates. The data dictionary surfaces any groups of metrics that share a definition under different names. Open the group, pick the one to keep as canonical, and Flynt removes the redundant entries.
  4. Inspect any metric. Open a metric's details to see its lineage, freshness and usage before you rely on it.

By design, the dictionary warns, it does not block. A warning is a prompt to think, not a wall. (If you want hard enforcement for specific people, for example a contractor whose work you always clean up, that is available as an opt-in, per-person setting rather than a blanket rule.)

Reusing a metric instead of redefining it​

The surest way to avoid duplicate definitions is to not re-write them. Once a metric is certified, you can reference it by name in a dataset's SQL with a {{metric:name}} token:

-- Instead of pasting the revenue definition again, reference it:
SELECT region, total
FROM {{metric:revenue}} AS r
JOIN regions USING (region_id)

At run time Flynt replaces {{metric:revenue}} with the metric's certified definition, wrapped as a subquery. One definition now feeds every dataset that references it, and editing the metric updates them all -- the reference stays live, it is not a one-time copy.

A few things worth knowing:

  • Names are forgiving. {{metric:Active Customers}} and {{metric:active_customers}} both resolve to the same metric.
  • It fails loudly, never quietly. If the metric does not exist, has no definition, or the resulting query is invalid, you get a clear error before anything runs. A reference can never silently produce a wrong number.
  • Today it works in datasets. Referencing metrics inside individual cards, and inside in-warehouse-cached datasets, is coming next.

Why this matters​

When everyone in the company pulls the same number from the same trusted definition, decisions speed up and stop being re-litigated. Flynt makes the definitions visible, keeps them from quietly fragmenting, and shows you the provenance of every figure. It is the difference between a pile of dashboards and a single source of truth your whole company can stand behind.