NULLBIT
NULLBIT
Blog
Author: ALFRED

Data Quality Management: Framework, Metrics and Checks

Data quality management explained: the dimensions to measure, metrics and formulas, validation techniques, a step-by-step framework and what changes for machine learning.

Data Quality Management: Framework, Metrics and Checks

Data Quality Management: Framework, Metrics and Checks

Sketch framing a data quality title

Data quality management (DQM) is the set of policies, roles, measurements and automated checks that keep your data fit for the decisions and systems that depend on it. It is not a one-off cleanup. It is a loop: define what “good” means for each use, measure it, fix problems at the source, and keep monitoring so the same problems do not return. This guide covers the dimensions, metrics, validation techniques and framework steps, plus what changes when the data feeds machine learning.

Nullbit
Make Data Work Better for Your Business
Nullbit builds custom software and AI solutions that help businesses optimize processes and support real-time decision-making.
Visit

TL;DR

  • Data quality management means defining quality per use case, measuring it with a small set of dimensions, and fixing root causes instead of patching symptoms.
  • Start with six dimensions: accuracy, completeness, uniqueness, consistency, timeliness and validity. Add others only when a real use case needs them.
  • Combine rule-based checks, statistical checks, cross-source comparison and human review. No single technique catches every error type.
  • A framework needs a policy, named owners, a measurement baseline, an issue process and continuous monitoring. Tools come after those.
  • For AI, add annotation quality, bias checks, drift monitoring and version control on top of the classic dimensions.

Table of Contents

What data quality management actually covers

Quality is relative. A customer table that is perfectly good for a monthly mailing may be useless for fraud detection, because the second use needs fresher, more precise and more complete records. DQM puts that context first: you decide what a use case requires, then you measure whether the data meets it.

In practice, DQM covers four activities:

  1. Defining quality expectations per dataset and use case.
  2. Measuring them continuously with metrics and rules.
  3. Resolving issues through a tracked process, ideally at the point where the data is created.
  4. Preventing repeats through validation at entry, better pipelines and clear ownership.

It sits next to, but is not the same as, data governance. Governance decides who owns what and which rules apply. DQM runs the measurements and fixes inside that structure. If you are building both, our guide to a data governance framework covers the roles and roadmap side.

Why it matters in money terms

Gartner research from 2020 puts the average cost of poor data quality at at least $12.9 million a year per organization. That figure is an average across large organizations, so treat it as an order of magnitude, not a forecast for your company. The costs usually show up as rework, failed reports, wrong customer communications, delayed decisions and, increasingly, unreliable AI output.

Gartner also lists the obstacles that keep organizations from fixing this: growing regulatory requirements such as GDPR and CCPA, inconsistency of data across sources (named as the most challenging problem), too few resources to scale a program, and a lack of ownership because business leaders do not see quality as their responsibility. The last one is the most common root cause we see: quality is treated as an IT chore, so nobody with authority over the business process fixes the source. A discussion of data responsibility as an organizational duty makes the same point from the ethics side.

The dimensions: what to measure

The “4 C’s”, “five measures” and “six pillars” you see in search results are all variations on the same idea: a list of dimensions that describe how data can be good or bad. There is no single official list, so pick one and stay consistent.

A practical starting set of six

The UK Government Data Quality Hub recommends six dimensions drawn from DAMA UK:

Dimension Question it answers Example failure
Accuracy Does the data reflect reality? A customer’s address is the one from five years ago
Completeness Is everything needed for the use present? ZIP code missing on an order
Uniqueness Does each thing appear once? The same client stored as two accounts
Consistency Do values agree within and across datasets? Billing says “active”, CRM says “churned”
Timeliness Is it available when expected? Yesterday’s sales load arrives at noon today
Validity Does it match the expected format, type and range? A birth date in the future

Longer lists for specialized needs

The US Health IT Playbook presents nine sample dimensions: accessibility, accuracy, completeness, coverage, conformity, consistency, integrity, timeliness and uniqueness. Its examples are concrete: a missing ZIP code fails completeness, and a birth date written as MMDDYYYY fails conformity when the standard is different. A catalog of 11 conformed dimensions adds currency, precision, lineage and representation. More dimensions give finer diagnosis, but also more to maintain. Begin with six, and add one only when a specific failure cannot be described by the existing ones.

Longer lists for specialized needs

Data quality metrics and how to calculate them

A dimension is a concept. A metric turns it into a number you can track. Most metrics are simple ratios:

  • Completeness rate = non-null values ÷ total values × 100. If 4,250 of 50,000 customer rows have no last name, completeness for that column is 91.5%.
  • Validity rate = values that pass the format, type and range rule ÷ total values × 100.
  • Accuracy rate = records that match a trusted source ÷ total records × 100.
  • Uniqueness rate = distinct entities ÷ total records × 100, after matching duplicates.
  • Timeliness = share of loads that arrive before the agreed deadline, or the age of the newest record.
  • Consistency rate = records where the same value agrees across two systems ÷ records compared.

The formulas are the easy part. The hard part is thresholds. A 98% completeness target makes sense for a key column that joins tables, and is irrelevant for an optional marketing field. Set each threshold by asking what the downstream decision can tolerate, then write it down next to the rule.

Two habits make metrics useful rather than decorative:

  • Score at several levels. Roll rule results into a column score, a table score and a score per data product, so a manager sees a single number and an engineer can drill into the failing rule.
  • Track trends, not snapshots. A table at 96% and falling is a bigger problem than one steady at 94%.

Data validation techniques that catch errors

A 2025 SciPy Proceedings paper by Edson Bomfim catalogs 23 common data quality problems and 22 detection methods, grouped into four strategies. They make a useful checklist for designing your own checks.

Rule-based and constraint enforcement

The workhorse. Not-null constraints, allowed value lists, regular expressions for formats, range limits, primary key uniqueness, referential integrity between tables, and business logic such as “ship date cannot precede order date”. Rules are cheap, explainable and catch most structural errors. Put them as close to the point of entry as possible.

Statistical and machine learning analysis

Rules only catch what you thought of. Statistical checks catch the rest: sudden changes in row counts, shifts in a column’s distribution, outliers, and unusual combinations of values. They are especially good at spotting silent breakage after an upstream change, where every rule still passes but the numbers are wrong.

Comparison with standards and other sources

Reconcile totals between a source system and the warehouse, compare against reference data such as official postal code lists, or check one system’s customer count against another’s. This is how you measure accuracy and consistency, since both require something to compare against.

Human and manual review

Sampling records and asking a domain expert whether they make sense still matters, particularly for free text, new data sources and edge cases. Use it to discover new rules, then automate those rules.

The paper’s broader conclusion is worth borrowing: treat data testing the way software teams treat software testing. It improves pipeline maintainability and trust in the dataset, and it improves the design of data-intensive systems.

Human and manual review

Building a data quality framework

Several published frameworks agree on the shape, even where they differ on vocabulary. A framework answers who decides, what is measured, how issues are handled and how the system keeps improving.

What the published frameworks have in common

  • A policy that states which quality dimensions matter to your organization. DAMA-NL’s framework treats this as the starting point and manages improvement actions (corrections, corrective measures, preventive measures) as SMART actions with a procedure to determine, record, plan, monitor, implement, evaluate and archive them.
  • Three views of quality. The Japan AI Safety Institute guidebook organizes the topic as a process view, a governance cycle view and a gateway view: the activities across the data lifecycle, the policies and roles that sustain it, and the quality characteristics themselves.
  • A staged rollout. A 2024 US business intelligence white paper proposes five phases: assessment and planning, establishing foundations, implementation, training and awareness, and continuous improvement.
  • A standards trend. ISO is developing ISO 8000-3, a document on requirements for establishing, implementing and continually improving data quality in an organization. It is listed as a work item, so check its status before you plan around it.

A six-step implementation you can start this quarter

  1. Pick two or three critical data assets. Choose those tied to revenue, compliance or a key AI use case. Boiling the ocean is how programs stall.
  2. Baseline them. Profile the data: null rates, distinct counts, value ranges, duplicates. Record the starting scores.
  3. Name owners. Each dataset gets a business owner who can change the process and a technical steward who maintains the checks.
  4. Write rules and thresholds. Translate each dimension into checks with a pass level and a severity.
  5. Run an issue process. Every failure is logged, ranked by business impact, traced to root cause and fixed at the source where possible. Track time to resolution.
  6. Monitor and expand. Schedule checks, alert the owners, review scores monthly and add the next datasets.

You will notice that tooling does not appear until step 4. Tools execute rules; they do not decide what the rules should be or who answers the alert.

Best practices that prevent recurring problems

The Government of New Brunswick’s data cleaning guidelines are a clear, public statement of good practice. Their core points:

  • Profile the data before you clean it, so you fix what is actually wrong.
  • Keep a secure backup of the raw data.
  • Define validation rules for formats, unique values, ranges and business logic.
  • Document every step so the work can be repeated and audited.
  • Automate repetitive tasks, but keep periodic manual review.
  • Treat cleaning as iterative, with a Plan-Do-Check-Act feedback loop.

Add three practices from our own project work:

  • Fix upstream first. A dropdown in the entry form beats a nightly script that repairs free text.
  • Make quality visible. Publish scores where data consumers look, such as the dashboard or catalog. Visible scores create pressure to fix.
  • Version your rules. Rules change as the business changes. Keep them in code, reviewed like any other change.

If your reporting layer is part of the problem, our business intelligence basics guide explains how definitions and models affect the numbers leaders see.

Automated data quality checks

Manual spot checks do not scale past a handful of tables. Automation should cover three layers:

  • At ingestion: schema checks, type checks and not-null checks that reject or quarantine bad records before they spread.
  • In transformation: tests inside the pipeline for uniqueness, referential integrity, row-count reconciliation and business rules.
  • In production: freshness monitors, volume and distribution anomaly detection, and alerts routed to the dataset owner rather than a shared inbox.

A failing check should do something defined: block the load, flag the record, or page a person, depending on severity. Alerts that nobody owns get muted within weeks. We cover how monitoring ties into the wider pipeline in system engineering work, because quality checks are only as dependable as the architecture they run in.

Data quality for machine learning and AI

Models learn from whatever they are given. Wrong labels, skewed samples and stale features produce confident but wrong output, and the error is harder to spot than in a report because there is no obviously broken number.

The Japan AI Safety Institute guidebook says AI use requires attention beyond accuracy, completeness and timeliness. Its list includes annotation quality management, bias and fairness checks, privacy and security measures (anonymization, masking, differential privacy), version control and drift management, handling of synthetic data, explainability, and operational monitoring. It also describes five approaches: make high-quality data, prevent low-quality data with validators and cleansing tools, build trust by verifying provenance, use data appropriately for AI training, and establish governance through a DataOps approach.

Translated into work for your team:

  • Labeling quality. Write clear labeling guidelines, have a second person label a sample, measure agreement between labelers and resolve disagreements by updating the guidelines, not just the labels.
  • Training versus production. Check that the data the model sees live looks like the data it trained on. Monitor drift on inputs, not only on model accuracy.
  • Lineage and versions. Be able to say which dataset version trained which model.
  • Representativeness. Check that important groups and edge cases appear in enough volume.

For a step-by-step view of preparing data before you build, see our guide to AI data readiness. Teams moving AI into live operations will also find that knowledge base governance for AI risk is the same discipline applied to the content a model retrieves. And the rules that sit around all of it are mapped in our piece on enterprise AI governance.

Our perspective

Most data quality programs fail for a reason that has nothing to do with tools: nobody owns the business process that creates the bad data. We start there. Before writing a single check, we ask who can change the form, the integration or the workflow that produces the error, and we put that person in the room.

The second thing we do differently is treat AI as part of the core design, not a layer added at the end. If a decision system or automation will consume the data, its needs set the quality thresholds from day one. That keeps the effort proportionate: strict where a wrong value costs money, relaxed where it does not.

We are technology-agnostic, so the checks fit the stack you already run rather than forcing a platform change. And we favor measurable outcomes: a score, a trend and a time to resolution that a leader can read without a data engineer translating.

Matija

Where we can help

If you want a quality baseline and a working set of automated checks rather than another slide deck, we build them as part of custom software, automation and AI projects. A proof-of-concept on one critical dataset is a practical first step, available from €5,000 one-off. Smaller, targeted automations, such as validation and alerting on a single pipeline, start from €3,000 one-off.

Nullbit

Larger programs run as an agile engagement or a fixed-price project. Details of both are on our cooperation page, and you can reach us any time through nullbit.co.

FAQ

What is data quality management?

It is the ongoing practice of defining, measuring, fixing and preventing data problems so data is fit for its intended use. It combines policy, ownership, metrics, validation rules and monitoring. It is a continuous cycle rather than a one-time cleanup.

What are the main dimensions of data quality?

The most widely used set is accuracy, completeness, uniqueness, consistency, timeliness and validity. Other frameworks add dimensions such as integrity, conformity, accessibility, currency and lineage. Choose a short list and apply it consistently across datasets.

How do you measure data quality?

Turn each dimension into a ratio, for example completeness as non-null values divided by total values times 100. Set a threshold per dataset based on what the downstream decision can tolerate, and track the score over time. Roll rule scores up to column, table and data product level for reporting.

Does data quality matter more for AI than for reporting?

It matters in both, but AI adds risks such as label errors, bias, drift and untraceable dataset versions. A bad value in a report is often visible; in a model it can quietly degrade predictions. Teams therefore add annotation checks, drift monitoring and version control to the usual dimensions.

Where should a company start?

Pick two or three datasets that drive revenue, compliance or an AI use case. Profile them, name an owner for each, write a few rules with thresholds and automate the checks. Expand once the first datasets show stable, improving scores.

Sources

Tags
data quality management
Stay ahead of the competition

Exclusive insights that drive change.

Get access to proven methodologies for digital growth, AI tool implementation, and AI product development.

  • Weekly digital strategy analyses
  • Advanced insights into AI trends and technology solutions

Your privacy is a priority. You can unsubscribe at any time.