Migration readiness

By Dzmitryi Kharlanau · Published · 23 min read

How to Detect Dataset Gaps Before SAP Migration Testing

Reviewed: 14 July 2026

Many SAP migration programmes treat testing as the first serious encounter between the target model and the source data.

The target structure is designed. Mappings are documented. Transformation routines are built. A migration file is prepared and sent into the first mock load.

Only then does the programme discover that:

These findings are often recorded as testing defects.

Many of them are not testing defects.

They are dataset-to-model gaps that existed before testing began.

The load did not create the problem. It only made the problem visible.

Our approach is to compare representative datasets with the approved model before formal migration testing. The goal is not to predict every possible load error. It is to remove avoidable uncertainty before the programme starts spending test cycles on problems that can be detected directly from the data and model.

Migration testing should verify implemented behaviour. It should not be the first time the programme checks whether the required data exists.

What we mean by a dataset gap

A dataset gap is a difference between what the approved migration model expects and what the current dataset can provide or prove.

This definition is broader than a missing column.

A dataset can contain every expected column and still have serious gaps.

For example:

We therefore treat dataset gaps as a family of problems rather than one technical check.

Why these gaps are found too late

The information needed to identify many gaps exists early in the programme.

The target team knows which attributes and relationships are expected.

The migration team has mapping workbooks.

Source owners can usually provide representative extracts.

Business teams know the most important local variations.

The problem is that these inputs are often reviewed separately.

The target model is reviewed as a design.

The mapping is reviewed as a document.

The source extract is profiled as a data-quality exercise.

No controlled comparison proves that all three remain aligned.

This separation creates a familiar sequence:

```text id="dsgap1" Target model approved ↓ Mappings completed ↓ Migration routines built ↓ Mock load executed ↓ Dataset gaps discovered


We prefer a different sequence:

```text id="dsgap2"
Target model defined
        ↓
Representative dataset profiled
        ↓
Model and dataset compared
        ↓
Gaps assigned and resolved
        ↓
Mappings and routines implemented
        ↓
Mock load verifies behaviour

The second approach does not eliminate defects.

It improves the quality of the defects reaching testing.

Instead of discovering that a required source field does not exist, the programme can spend the test cycle on whether the agreed treatment works correctly.

A model is an expectation

A migration model expresses expectations about the data.

It may state that a Business Partner requires:

The dataset provides evidence about whether those expectations can be met.

The two should be compared explicitly.

This sounds obvious, but many mapping processes compare fields only at design time:

```text id="dsgap3" Expected source field → Target field


The programme also needs to compare the current data:

```text id="dsgap4"
Expected field
        ↕
Actual column presence
Actual completeness
Actual values
Actual formats
Actual relationships
Actual population coverage

A mapping tells us what should happen.

A profile tells us whether the current population supports it.

The major types of dataset gaps

We use several categories because each type requires a different response.

1. Structural gaps

A structural gap exists when the dataset does not contain the columns, files or relationships expected by the model.

Examples include:

Structural checks are the simplest to automate and should happen immediately when a new extract arrives.

They answer questions such as:

A mock load should not be the mechanism used to discover that a required file was omitted.

2. Completeness gaps

A column can exist while providing insufficient data.

Completeness must be measured against the relevant population and context.

For example:

An overall completeness percentage can conceal the operational gap.

We therefore profile completeness by dimensions such as:

The relevant question is not:

How complete is this column?

It is:

Is this column complete enough for the population and context we intend to migrate?

3. Value-domain gaps

A value-domain gap exists when the dataset contains values that are not covered by the approved target list or value mapping.

Examples include:

A value-mapping workbook may appear complete because every expected source code has a target.

The real dataset may contain additional values that were never included in the workbook.

This is why we calculate value coverage against actual data:

```text id="dsgap5" Observed source values ↓ Normalisation ↓ Mapped values Unmapped values Ambiguous values Deprecated values


The programme then decides whether unmatched values should be:

- added to the mapping;
- corrected in the source;
- normalised;
- rejected;
- defaulted;
- excluded;
- reviewed manually.

That decision should happen before the load routine encounters the value.

# 4. Format and representation gaps

Some values contain the right information in a form that does not match the target expectation.

Examples include:

- dates stored in several formats;
- leading zeros removed from identifiers;
- decimal separators varying by country;
- multiple values combined in one field;
- names and addresses stored in unstructured text;
- country codes mixed between ISO forms and local abbreviations;
- boolean values represented as `Y/N`, `1/0`, `true/false` and blanks;
- units stored inside numeric fields;
- invalid character encoding;
- text truncated by the extraction format.

These problems are often described as technical cleansing.

Some are technical. Others require business interpretation.

Removing spaces from a tax identifier may be straightforward. Deciding which of two values in one field is authoritative is not.

We distinguish between:

- syntactic normalisation;
- semantic transformation;
- business decision.

Only the first category should be treated as automatically safe.

# 5. Context gaps

A value can be present but attached to the wrong or incomplete business context.

This is especially important in SAP master data.

Examples include:

- a customer attribute available centrally but required by sales area;
- supplier data available generally but missing by purchasing organisation;
- material data complete at basic level but absent for one plant;
- a local field included without the country or company-code condition that explains it;
- the same source value meaning different things in different organisational units.

A flat dataset may hide these distinctions.

The programme should compare data at the same level at which the target model expects it.

For example:

```text id="dsgap6"
Customer
 ├── Central data
 ├── Company code A
 ├── Company code B
 ├── Sales area 1
 └── Sales area 2

A customer record is not complete merely because the central row exists.

If the go-live scope includes Sales Area 2, the required organisational data must also be present.

6. Relationship gaps

Enterprise master data is not a collection of independent rows.

Important meaning exists in relationships:

A relationship gap exists when:

Relationship gaps are frequently discovered late because teams profile files column by column.

We also need cross-file and cross-object checks.

For example:

A file can be internally clean and still be unusable as part of the full model.

7. Identifier gaps

Identifiers hold the migration together.

A dataset gap exists when identifiers are:

Identifier problems affect more than loading.

They affect:

Before testing, the programme should prove that each record can be identified consistently across:

If the programme cannot trace a rejected target record back to its source record, the migration process is not operationally ready.

8. Rule-coverage gaps

A dataset may contain values, but the programme may not know whether they satisfy the approved rules.

Examples include:

Rule-coverage analysis compares the dataset with business and target rules before the full migration routine is executed.

Useful checks include:

This is not intended to recreate the full SAP or MDG validation engine.

It identifies whether the dataset contains populations likely to conflict with known rules.

9. Scope gaps

A dataset may be high quality but incomplete relative to the migration scope.

Examples include:

Scope gaps are difficult to detect through field profiling alone.

The programme needs population expectations:

We compare actual coverage with those expectations.

A dataset of 100,000 technically valid records may still be incomplete if 20,000 in-scope records were never extracted.

10. Temporal gaps

Data changes during the programme.

A profile from three months ago does not prove that the current extract is ready.

Temporal gaps include:

Every material dataset version should therefore be compared with:

  1. the current approved model; and
  2. the previous relevant dataset version.

The first comparison finds model gaps.

The second finds change.

What to profile before migration testing

A useful pre-test profile should provide more than row counts.

For each dataset, we recommend capturing:

Not every field requires the same depth.

Critical and high-risk attributes deserve more detailed analysis.

Examples include:

The profiling strategy should be risk-based rather than mechanically identical for every column.

The model must be precise enough to compare

Automated gap detection is only as good as the model.

The model should tell us:

If the target model is only a list of field names, the comparison will be shallow.

For example, this requirement is difficult to validate:

```text id="dsgap7" Tax Number — mandatory


A more useful model states:

```text id="dsgap8"
Attribute: Tax Number
Context: Business Partner tax category DE1
Population: Organisations registered in Germany
Requirement: Mandatory unless an approved exemption applies
Source: LEGACY_BP.DE_TAX_ID
Normalisation: Remove spaces, uppercase letters
Invalid treatment: Reject for review
Owner: German Tax Data Owner

The second representation allows the programme to profile the relevant population and classify gaps meaningfully.

Do not confuse detection with resolution

A gap report does not decide automatically what the programme should do.

Suppose 15% of supplier records lack a required classification.

The correct response could be:

The tool can identify the population and affected model objects.

The programme must make the decision.

We deliberately separate:

```text id="dsgap9" Detection → Classification → Ownership → Decision → Implementation → Verification


Skipping classification creates a defect backlog filled with different kinds of problems.

Skipping ownership creates reports that nobody acts on.

Skipping verification allows the same gap to return in the next extract.

## How to classify gaps

We use a practical classification model.

### Source gap

The required data is absent or unreliable in the source.

### Extraction gap

The source contains the data, but the extract does not.

### Model gap

The approved model does not adequately represent the business requirement or source reality.

### Mapping gap

The source and target exist, but the transformation is missing or ambiguous.

### Value-mapping gap

Observed source values are not covered.

### Context gap

The rule or value cannot be interpreted without missing organisational or business context.

### Relationship gap

Required links between records cannot be constructed.

### Governance gap

A technical treatment is possible, but no owner has approved it.

### Test-evidence gap

The rule is defined and implemented but has not been verified with representative data.

This classification helps management understand whether the migration team can resolve the issue alone.

Often it cannot.

## A practical pre-testing process

We recommend a repeatable cycle rather than a one-time data-quality assessment.

# Step 1: Freeze a model baseline

Identify the approved model version against which the dataset will be checked.

The baseline should include:

- target attributes;
- required contexts;
- source expectations;
- value lists;
- mappings;
- relevant rules;
- scope.

Without a baseline, gap results cannot be interpreted consistently.

# Step 2: Register the dataset

Record:

- source;
- extraction date;
- filters;
- included populations;
- file structure;
- responsible owner;
- intended migration wave.

A file called `Customer_Final.xlsx` is not sufficient dataset identity.

# Step 3: Run structural checks

Check:

- expected columns;
- unexpected columns;
- data types;
- required keys;
- duplicate column names;
- related-file availability;
- join fields.

These checks should produce immediate blockers where appropriate.

# Step 4: Profile critical values

Measure:

- completeness;
- uniqueness;
- distributions;
- patterns;
- lengths;
- invalid references;
- context variation;
- observed source codes.

Do not wait until every field is perfectly modelled. Start with the fields carrying the greatest risk.

# Step 5: Compare the profile with the model

Identify:

- missing expected fields;
- uncovered values;
- insufficient completeness;
- invalid relationships;
- context mismatches;
- source assumptions contradicted by the data;
- target requirements unsupported by the source.

# Step 6: Classify and assign gaps

Each material gap receives:

- type;
- affected model objects;
- affected population;
- severity;
- evidence;
- owner;
- proposed treatment;
- required decision date.

# Step 7: Review with the correct roles

Not every gap needs a programme-wide meeting.

Route gaps appropriately:

- source owners for extract and quality issues;
- business owners for definitions and acceptable treatments;
- MDG architects for target-model and validation questions;
- migration architects for transformations;
- programme management for scope and accepted risk.

# Step 8: Update the model or treatment

If the gap reveals that the model assumption was wrong, update the model through controlled review.

Do not hide the discovery inside migration code.

# Step 9: Re-profile the next dataset

A resolved gap should be verified against a new extract or corrected population.

Closing the ticket is not evidence that the data changed.

## A worked example: Business Partner roles

Suppose the target design requires every migrated organisation to receive one or more Business Partner roles.

The mapping workbook identifies legacy customer and supplier indicators.

The first dataset profile finds:

- 70% of records contain a clear customer indicator;
- 20% contain both customer and supplier indicators;
- 5% contain neither;
- 3% contain values not included in the mapping;
- 2% have conflicting indicators across source files.

Before testing, the programme already has several questions:

- Are records without indicators in scope?
- Can roles be derived from transaction history?
- Should records with both indicators receive both roles?
- Which source is authoritative when indicators conflict?
- Are unexpected values data errors or legitimate local codes?
- Does the target workflow support the resulting roles?
- Will duplicate detection treat the role combinations correctly?

A mock load could expose these questions through failures and unexpected records.

A pre-test gap analysis exposes them as model and governance decisions.

The programme can then test an approved rule rather than improvise during defect triage.

## Another example: customer sales-area data

The central customer dataset contains 50,000 records.

All required central fields are present.

The programme reports good data readiness.

A separate analysis of sales-area assignments shows:

- 12,000 customers lack an expected sales-area record;
- 4,000 use sales organisations outside the target scope;
- 2,500 have incomplete distribution-channel data;
- several legacy systems use a composite key not included in the extract.

The central dataset is structurally complete.

The operational business object is not.

This is why dataset readiness must follow the model structure, including organisational extensions and relationships.

## How early gap detection changes testing

When gaps are detected before testing, the first mock load becomes more useful.

Instead of spending the cycle on basic surprises, the programme can verify:

- transformation correctness;
- SAP and MDG validation behaviour;
- load sequencing;
- matching and duplicate handling;
- key mapping;
- relationship creation;
- workflow behaviour after load;
- replication;
- reconciliation;
- performance and volume.

Testing becomes evidence about the implemented migration process.

It no longer carries the full burden of discovering what the source data contains.

## What managers should ask before approving a mock load

Managers do not need to inspect field-level profiles.

They should ask:

1. Which approved model baseline was used for the comparison?
2. Which representative datasets were profiled?
3. Are all countries and source systems in the wave represented?
4. Which required fields are absent from the extract?
5. Which critical fields have insufficient completeness?
6. Which observed values are not covered by approved mappings?
7. Which relationships cannot be constructed?
8. Which target requirements are unsupported by source data?
9. Which gaps still require business or architecture decisions?
10. Which accepted gaps will deliberately enter testing?
11. Who owns each unresolved population?
12. Will the next extract prove that agreed corrections were made?

A mock load may proceed with known gaps.

The point is that the gaps should be known, owned and intentionally included—not discovered accidentally.

## Severity should reflect business impact

Not every unexpected value should block testing.

We use severity based on questions such as:

- Does the gap prevent loading?
- Does it create an incorrect business meaning?
- Does it affect a critical process?
- Does it create regulatory or compliance exposure?
- Does it break relationships?
- Does it affect a large population?
- Can it be corrected after go-live?
- Is the treatment reversible?
- Does it hide behind a valid-looking default?

A missing optional description is not equivalent to a missing legal identifier.

An unmapped value affecting three obsolete records is not equivalent to an unmapped classification used by half the active customer population.

Gap counts without severity and population context can mislead management.

## Avoid the “clean data” target

Programmes sometimes state that the dataset must be clean before testing.

This sounds responsible but is too vague to govern delivery.

No large legacy dataset is perfectly clean.

We prefer explicit readiness rules:

- all required columns must be present;
- no critical reference relationships may be broken;
- mandatory attributes must meet an agreed completeness threshold;
- all observed critical codes must have an approved treatment;
- unresolved high-severity gaps must have an owner and decision;
- accepted exceptions must be identifiable in the dataset;
- the loaded population must be reconcilable to the source.

These criteria can be reviewed and evidenced.

“Clean enough” cannot.

## Where Martenweave fits

We built Martenweave to compare dataset reality with model expectations before the programme reaches migration testing.

The current SAP migration scenario supports:

- explicit source and target field endpoints;
- contextual attribute definitions;
- source-to-target mappings;
- deterministic model validation;
- CSV and XLSX profiling;
- dataset-to-model gap detection;
- lineage and impact analysis;
- issues, decisions and controlled change proposals.

The broader product model includes canonical Markdown and YAML files, generated SQLite and JSONL indexes, structured queries, ownership and health reports, review exports and a PatchProposal-to-ChangeRequest lifecycle.

A Martenweave workflow can operate as follows:

```text id="dsgap10"
Canonical model
      +
Registered dataset
      ↓
Structural and profile analysis
      ↓
Dataset-to-model comparison
      ↓
Gap objects linked to affected fields and rules
      ↓
Owner review and decision
      ↓
Model patch or data remediation
      ↓
Repeatable validation

The dataset does not become the source of model truth.

The model does not pretend that source data is cleaner than it is.

The comparison between them creates the evidence.

Why canonical model files matter here

A repeatable gap check requires a stable model baseline.

If expectations live only in workbook comments, the programme cannot reliably compare one dataset version with another.

Canonical model objects make it possible to state:

The generated profile and gap results can be rebuilt when:

This is more useful than a static data-quality report whose assumptions are no longer clear.

AI can assist, but it should not decide

AI may help with:

These are proposals.

An unexpected source value may look similar to a target code and still have a different business meaning.

A missing field may appear derivable but require business approval.

A relationship may look inferable statistically but be unsafe to create automatically.

Our rule remains:

AI may propose a treatment. Deterministic checks verify structure. Responsible people approve changes to the model or migration rule.

What Martenweave does not replace

Martenweave does not replace:

Its role is narrower.

It connects the approved model with representative datasets so gaps can be found, classified and traced earlier.

For small migrations, a profiling script and disciplined workbook may be sufficient.

The need for a model-driven gap layer grows when the programme has:

Common mistakes

Profiling without a model

The team produces distributions and completeness percentages but cannot say whether the results are acceptable.

A profile becomes useful when it is compared with explicit expectations.

Comparing only column names

Presence does not prove completeness, valid values, correct context or relationship coverage.

Profiling only once

Every significant extract should be checked against the current model.

Reviewing global averages

Gaps should be analysed by the populations that matter operationally.

Closing gaps through undocumented code

If a transformation changes the treatment, the model and decision record should be updated.

Treating every anomaly as a defect

Some anomalies are valid exceptions. They still need classification and evidence.

Using defaults to hide missing data

A populated target field may remain semantically empty.

Sending all gaps to the migration team

Many gaps require business, source-system, architecture or governance decisions.

Starting formal testing with unknown data scope

The programme should know which populations the dataset represents before interpreting test results.

A minimum pre-test gate

Before formal migration testing, we recommend confirming:

Dataset identity

Structure

Content

Relationships

Model alignment

Governance

A dataset does not need to be perfect to pass the gate.

It needs to be understood well enough that testing results can be interpreted reliably.

Our conclusion

Dataset gaps should not be treated as surprises produced by migration testing.

Many can be detected earlier by comparing three things that the programme already has:

The comparison needs to go beyond column presence.

It should examine:

Our position is straightforward:

Testing should confirm that the agreed migration treatment works. It should not be the first point at which the programme discovers that the treatment is impossible.

Finding gaps early does not make difficult decisions disappear.

It gives the correct people time to make them before technical defects, cutover pressure and programme deadlines narrow the available options.

That is how earlier gap detection saves time.

Not by producing fewer reports, but by preventing the programme from building and testing assumptions that the data could have disproved weeks earlier.

About the authors

We are Metalhatscats, the team behind Martenweave.

We work on practical model governance for SAP migration, MDG, MDM and AMS programmes. Our focus is helping architects and delivery teams compare model expectations with actual data, identify gaps before they become load defects and preserve the evidence needed to make controlled decisions.

Sources and notes

This article was reviewed on 14 July 2026.

SAP describes SAP Master Data Governance as supporting governed master-data models, validated values, quality monitoring, matching and consolidation, workflow-based changes and auditability. These capabilities provide the operational governance context into which migrated master data must fit.

Martenweave’s SAP migration documentation describes contextual model objects, source-to-target mappings, deterministic validation, CSV and XLSX profiling, dataset gap detection, lineage, impact analysis and human-reviewed model proposals.

The current Martenweave product documentation lists canonical model files, generated indexes, dataset profiling, health and analysis reports, spreadsheet review flows and controlled PatchProposal and ChangeRequest lifecycles.

Martenweave is an independent project and is not affiliated with or endorsed by SAP. SAP, SAP S/4HANA and SAP Master Data Governance are trademarks or registered trademarks of SAP SE or its affiliates.

Related articles