Lineage and impact

Why Table-Level Lineage Is Not Enough for Master Data

By Dzmitryi Kharlanau · Published · 17 min read

A migration architect opens the lineage view and sees:

Contents

Reviewed: 14 July 2026

A migration architect opens the lineage view and sees:

Legacy ERP Customer
→ Migration Staging
→ SAP S/4HANA Customer

The diagram looks complete.

Every system is connected.

Every table has an upstream and downstream dependency.

The programme can answer:

Then a defect appears.

Customer Group is correct for some customers and wrong for others.

The table-level lineage still says:

Legacy customer table
→ customer staging table
→ KNVV

That does not explain:

Table-level lineage proves that a dataset moved.

It does not prove that a master-data value was interpreted correctly.

Master-data lineage needs to describe identity, granularity, context, transformation and governance—not only movement between tables.

OpenLineage’s current specification makes the same underlying distinction. Dataset-level lineage represents dependencies between datasets, while column-level lineage adds which input fields produce which output fields and how the transformation works. Its specification distinguishes direct value derivation from indirect influences such as filters, joins and conditional logic.

For SAP migration and MDM, even column-level technical lineage is only part of the answer.

The organisation also needs to know:

This is where a model registry such as Martenweave must go beyond ordinary table lineage.

---

What table-level lineage does well

Table-level lineage is useful.

It answers important operational questions:

Which source table feeds this staging table?

Which job writes this target table?

Which downstream datasets depend on this extract?

Which pipelines may be affected if a table is removed?

In a migration landscape, this can reveal paths such as:

LEGACY_CUSTOMER
→ STG_CUSTOMER
→ LOAD_CUSTOMER
→ SAP_KNA1

or:

LEGACY_CUSTOMER_SALES
→ STG_CUSTOMER_SALES
→ LOAD_CUSTOMER_SALES
→ SAP_KNVV

This helps with:

OpenLineage’s object model is designed around Datasets, Jobs and Runs. It treats a Dataset as a discrete collection such as a database table or storage object and connects datasets through jobs that create or transform them. It also separates design-time metadata from runtime observations.

This is strong infrastructure lineage.

It becomes insufficient when the problem is not:

Did the customer table arrive?

but:

Did the correct Customer Group reach the correct Customer Sales Area under the correct rule?

---

Master data is not one flat record

Transactional datasets are often analysed as streams of events or rows.

Master data is structurally different.

A Business Partner may have:

The relevant unit is not always “the customer table.”

For example:

Customer Name

may exist centrally.

Payment Terms

may exist at company-code or sales-area level depending on the business context.

Customer Group

may exist per Sales Area.

Partner Function

may depend on Sales Area and partner-role structure.

A table-level graph may show that KNVV came from a source customer-sales table.

It does not show whether the source extract preserved:

Customer
Sales Organisation
Distribution Channel
Division

Without those keys, the target field lineage is incomplete even when both tables are connected.

---

Granularity is part of lineage

Suppose a legacy system stores one Customer Group per customer.

SAP expects Customer Group per Customer Sales Area.

The source and target tables may be linked correctly:

LEGACY_CUSTOMER
→ KNVV

The granularity is not.

One source value may be:

The lineage must therefore state:

Source grain:
one value per Customer

Target grain:
one value per Customer Sales Area

and then explain the transition.

A useful path might be:

Legacy Customer Segment
+
Legacy Sales Area Assignment
→ contextual Customer Group Mapping
→ Customer Group
→ KNVV-KDGRP

Table-level lineage cannot express this accurately because both source grain and target grain are hidden inside dataset structure.

---

A table contains several unrelated semantic paths

One source table may contain hundreds of fields.

Those fields may feed different target objects through different logic.

For example:

LEGACY_CUSTOMER

could supply:

A single edge:

LEGACY_CUSTOMER → SAP_CUSTOMER

implies a uniform relationship that does not exist.

The real model may be:

NAME1
→ direct copy
→ Business Partner Name

SEGMENT_CODE
+ SALES_UNIT
→ enrichment
→ Customer Group

PAYMENT_CLASS
→ lookup
→ Payment Terms

LEGACY_STATUS
→ filter only
→ record eligibility

OLD_RISK
→ rejected as non-authoritative

Each field follows a separate semantic and governance path.

---

Table lineage hides transformation diversity

Two tables can be connected through one job, but every target field may use a different transformation type.

OpenLineage’s column-level lineage facet distinguishes:

SAP master-data migrations need additional business-oriented distinctions:

direct copy
value conversion
reference-data lookup
organisational enrichment
manual assignment
default
rejection
split
merge
source-authority selection

Consider two fields in the same target table:

KNVV-KDGRP
KNVV-VSBED

Both may be populated by the same migration job.

Customer Group may require a contextual lookup.

Shipping Condition may be copied directly.

At table level, both appear identical:

STG_CUSTOMER_SALES → KNVV

At field level, they have different risk, ownership and test requirements.

---

Table lineage does not show business meaning

A technical field can be connected perfectly and still represent the wrong concept.

Example:

CRM_SEGMENT
→ KNVV-KDGRP

Technically, the source field populates the target.

Semantically, the questions remain:

A lineage graph that connects only columns may tell the truth about implementation while hiding a false business assumption.

Martenweave needs a semantic layer between source and target:

FEP-CRM-CUSTOMER-SEGMENT
→ Mapping
→ ATTR-CUSTOMER-GROUP
→ FEP-S4-KNVV-KDGRP

The business Attribute acts as the governed meaning of the path.

The Mapping explains whether the source actually satisfies that meaning.

---

Physical field and business attribute are different objects

This distinction is essential.

KNVV-KDGRP

is a physical SAP field endpoint.

Customer Group

is a governed business Attribute.

The Attribute can survive:

The endpoint may not.

When these are collapsed, a field change looks like a business-model change.

When they are separate, the model can state:

Business meaning:
unchanged

Physical SAP implementation:
changed

Martenweave’s current public example already separates the Customer Group business Attribute from the physical KNVV-KDGRP FieldEndpoint and supports trace and impact operations across mappings, decisions and evidence.

---

Table lineage cannot represent rejected mappings

A migration team may investigate a possible source and conclude:

Legacy Classification must not populate Supplier Risk.

That conclusion is valuable.

A normal lineage system records positive paths:

source → target

It usually does not preserve:

source must not map to target

Without the negative knowledge, a later team or AI agent may rediscover the same apparent similarity and propose the rejected mapping again.

A model-governance layer should preserve:

This is not table dependency.

It is institutional model memory.

---

Table lineage does not show authority

Several source tables may contain a value called RISK_CLASS.

Only one may be authoritative.

Another may contain:

The graph may show all three feeding the migration environment.

That does not answer:

Which source is approved for the target Attribute?

The lineage model needs source authority by:

Example:

Global Risk System:
authoritative for final Supplier Risk

Local ERP:
temporary fallback for historical suppliers

Review Workflow:
authoritative for Review Status, not Risk

Table-level edges cannot express these distinctions clearly.

---

Table lineage does not show applicability

A Mapping may be valid only when:

Country = PT
Supplier Type = Organisation
Purchasing Organisation belongs to Region South

A table-level edge implies the transformation applies to every row moving between the tables.

The real path is conditional.

A field-level model must connect both direct and indirect inputs.

Tax Status
→ direct value input

Country
→ conditional input

Supplier Type
→ applicability input

OpenLineage’s specification similarly distinguishes direct dependencies, where the output derives from an input value, from indirect dependencies, where an input controls selection or calculation without contributing its value directly.

For master data, these conditional fields are often as important as the primary source field.

---

Table lineage hides keys

A migration target field is not meaningful without its target record.

For KNVV-KDGRP, the relevant record is determined by:

KUNNR
VKORG
VTWEG
SPART

The value path alone is insufficient.

The migration must also establish:

Legacy Customer ID
→ SAP Customer Number

Legacy Sales Unit
→ SAP Sales Organisation

Legacy Channel
→ SAP Distribution Channel

Legacy Division
→ SAP Division

If the key lineage is wrong, the right value can land on the wrong record.

Table-level lineage usually treats keys as part of the dataset rather than separate governed paths.

For migration assurance, key lineage must be explicit.

---

Table-level completeness can be misleading

Suppose every source table and target table has a documented connection.

A programme reports:

Lineage coverage: 100%

But among 200 target fields:

The table graph is complete.

The migration lineage is not.

A meaningful coverage scorecard should separate:

MeasureCoverage
Target datasets with upstream dataset100%
Target fields with identified source82%
Target fields with approved Mapping69%
Transformations classified61%
Critical paths with test Evidence44%
Contextual fields with complete key lineage38%

The first number alone creates false confidence.

---

Master data includes rule lineage

Some target values are not simply transformed.

They are permitted, required or rejected through Rules.

Example:

Supplier Risk is mandatory before activation
when Supplier Category = STRATEGIC.

The lineage question is not only:

Which source field populates Supplier Risk?

It is also:

A complete path may be:

Risk Score
→ Supplier Risk Mapping
→ Supplier Risk Attribute
→ SAP Risk Field

Supplier Category
→ applicability condition
→ Risk Required Rule

Table-level lineage normally excludes this control layer.

For MDM and migration readiness, the control layer is part of the real dependency.

---

Rule implementation may differ from rule intent

The canonical Rule may state:

Missing Supplier Risk blocks activation.

The SAP implementation may issue only a warning.

The source-to-target table path is unaffected.

Table-level lineage reports no problem.

A model registry can record:

Canonical Rule
→ implemented by SAP validation
→ observed behaviour

The difference becomes implementation drift.

This is a governance and control issue, not a data-movement issue.

---

Master data includes lifecycle lineage

A value can change meaning depending on its lifecycle.

For example:

PENDING
CLEARED
REJECTED

may describe review status.

LOW
STANDARD
HIGH

may describe final risk classification.

If a source table stores both in one column, the migration may need to split them into two target Attributes.

At table level:

LEGACY_SUPPLIER → SAP_SUPPLIER

looks simple.

At model level:

Legacy Supplier Classification
→ semantic split
├── Supplier Risk
└── Supplier Review Status

This requires:

A table edge cannot represent the decomposition.

---

One table may contain several business entities

A legacy table often mixes concepts that the target separates.

For example, one customer table may contain:

The target may distribute these across several objects and contexts.

Table-level lineage may show one source table feeding five SAP targets.

This still leaves unanswered:

The correct lineage must follow model objects, not only storage containers.

---

Several tables may implement one business concept

The reverse problem is also common.

One business Attribute may be assembled from:

Example:

Customer Group

may depend on:

CRM customer segment
legacy sales organisation
regional conversion table
manual exception list

The table graph shows four upstream datasets.

It does not explain their roles.

Field-level semantic lineage can distinguish:

CRM Segment:
value input

Sales Organisation:
context input

Conversion Table:
lookup implementation

Exception List:
temporary override

---

Table lineage does not explain defaults

Defaults are particularly dangerous in migration.

A target table can be fully populated even when the source does not support the value.

Example:

Missing Customer Group
→ default 01

A table-level reconciliation may report:

Target row count matches.
Target field completeness is 100%.

The business risk remains hidden.

A governed lineage path should show:

The correct lineage is not:

Source Customer Table → KNVV

It is:

No authoritative source
→ approved temporary default
→ Customer Group
→ KNVV-KDGRP

---

Table lineage does not explain manual work

Many master-data migrations contain manual steps:

Technical lineage often omits them because they are not jobs in the orchestration platform.

The resulting graph falsely suggests automation.

A truthful path might be:

Legacy Tax Status
→ legal review workbook
→ approved Tax Exemption decision
→ migration Mapping
→ SAP Tax Classification

The workbook should not become canonical model truth.

The manual step should still be visible as Evidence or an implementation activity.

---

Table lineage does not expose unresolved ambiguity

Suppose a target field is populated, but two teams disagree on the source meaning.

One says:

CRM Segment is authoritative.

Another says:

Legacy Sales Classification is authoritative.

The pipeline currently uses CRM.

Table-level lineage reports:

CRM table → SAP table

This makes an implementation choice look like an approved semantic decision.

A model registry should represent:

This distinction prevents runtime behaviour from silently becoming governance truth.

---

Table lineage is weak for impact analysis

Consider a change to Customer Group.

Table-level impact may identify:

KNVV
customer staging table
migration customer job

A real impact assessment may need:

The change is not a table change.

It is a governed concept change with table consequences.

Martenweave’s pipeline explicitly runs lineage and impact analysis after validation and index generation, using canonical objects rather than an independently edited technical graph.

---

The right model is layered lineage

The solution is not to discard table-level lineage.

It is to place it inside a layered model.

System layer

Legacy CRM
→ Migration Platform
→ SAP S/4HANA

Useful for architecture and ownership.

Dataset layer

CRM_CUSTOMER
→ STG_CUSTOMER
→ LOAD_CUSTOMER_SALES
→ KNVV

Useful for pipeline operations.

Field layer

SEGMENT_CODE
+ SALES_UNIT
→ KDGRP

Useful for transformation and defect investigation.

Semantic layer

Customer Segment
→ approved Mapping
→ Customer Group

Useful for business meaning.

Governance layer

Decision
→ Mapping
→ Rule
→ Evidence

Useful for approval and traceability.

Runtime layer

Extract 2026-07-12
→ transformation run 184
→ mock load 3

Useful for observed execution.

No single layer is enough.

Together, they create usable lineage.

---

Not every field needs full semantic lineage

A programme should not model every technical column with equal depth.

Some fields may need only:

source column
→ direct copy
→ target column

Others require the complete governance chain.

Prioritise fields that are:

This prevents lineage work from becoming another metadata-collection programme with no delivery outcome.

---

A practical escalation rule

Use table-level lineage as the starting point.

Escalate to field-level and semantic lineage when at least one of these conditions applies:

The target field is critical.

The transformation is not a direct copy.

Source and target grain differ.

Several sources compete.

A default is used.

A manual step exists.

A Rule affects applicability or requiredness.

A Decision is needed.

A defect cannot be explained at table level.

The field changes across releases.

Downstream consumers depend on its meaning.

This keeps the approach proportionate.

---

A Customer Group example

Table-level view

CRM_CUSTOMER
→ CUSTOMER_STAGING
→ SAP_KNVV

Useful information:

Missing information:

Field-level view

CRM_CUSTOMER.SEGMENT_CODE
+
CRM_CUSTOMER.SALES_UNIT
→ CUSTOMER_STAGING.CUSTOMER_GROUP
→ SAP_KNVV.KDGRP

Better, but still incomplete.

Governed lineage

FEP-CRM-CUSTOMER-SEGMENT
+
FEP-LEGACY-SALES-AREA
→ MAP-CUSTOMER-GROUP-ENRICHMENT
→ ATTR-CUSTOMER-GROUP
→ FEP-S4-KNVV-KDGRP

Connected governance:

DEC-CUSTOMER-GROUP-SOURCE-017
→ authorises Mapping

RULE-CUSTOMER-GROUP-REQUIRED
→ governs Attribute

EVID-MOCK-LOAD-3-KDGRP
→ verifies target result

This path supports investigation and controlled change.

---

A Supplier example

Table-level view

LEGACY_SUPPLIER
→ SUPPLIER_STAGING
→ SAP_SUPPLIER

Actual model problem

The legacy field contains:

LOW
MEDIUM
HIGH
UNDER_REVIEW

The target separates:

Supplier Risk:
LOW
MEDIUM
HIGH

Supplier Review Status:
PENDING
CLEARED
REJECTED

The correct lineage is a semantic split:

LEGACY_SUPPLIER.CLASSIFICATION
→ classification interpretation
├── ATTR-SUPPLIER-RISK
└── ATTR-SUPPLIER-REVIEW-STATUS

Table lineage cannot explain why one source field produces two governed target concepts.

---

A Product example

Table-level view

LEGACY_MATERIAL
→ PRODUCT_STAGING
→ MARA/MARC

Missing granularity

A legacy field may be central.

The SAP target may be plant-specific.

Example:

Planning Group

may require plant context.

The complete path needs:

Legacy Product
+
Legacy Site
→ Plant mapping
→ Product Plant Attribute
→ SAP MARC field

The table path alone hides the central-to-plant expansion.

---

Table-level lineage is still useful for discovery

A table graph can help identify where to investigate first.

For example:

  1. find datasets feeding KNVV;
  2. identify jobs writing the table;
  3. inspect candidate source tables;
  4. extract field-level mappings;
  5. connect fields to business Attributes;
  6. add Decisions and Evidence.

This is a sensible bottom-up workflow.

The mistake is stopping after step 1 and calling the lineage complete.

---

Automated extraction has limits

Technical tools may extract lineage from:

This can produce valuable field dependencies.

It may still miss:

Automation should create candidate technical lineage.

The governance layer requires controlled model objects and human review.

---

AI can bridge documents and technical lineage

AI can help connect:

It might suggest:

SEGMENT_CODE appears to feed KDGRP through lookup CUSTOMER_GROUP_MAP.

The proposal should remain reviewable.

Martenweave’s current operating principle is that agents propose, validators verify, humans approve and Git records the accepted change.

This is especially important for lineage.

One false edge can distort impact analysis across many objects.

---

A layered lineage query

A useful Martenweave query for FEP-S4-KNVV-KDGRP should return something like:

Physical target:
S4 KNVV-KDGRP

Business Attribute:
Customer Group

Entity:
Customer Sales Area

Upstream source:
CRM Customer Segment

Context inputs:
Sales Organisation
Distribution Channel
Division

Transformation:
Conditional enrichment

Mapping status:
Approved

Decision:
Direct equivalence rejected

Rule:
Required before Sales Area activation

Evidence:
Mock Load 3 validation

Open gap:
ERP_B lacks Sales Area context

This is much more actionable than:

Upstream table:
CRM_CUSTOMER

---

A minimum viable improvement over table lineage

Martenweave does not need to recreate a full enterprise lineage platform.

The smallest useful extension is to require, for critical target fields:

  1. target FieldEndpoint;
  2. business Attribute;
  3. source FieldEndpoint or explicit missing source;
  4. Mapping type;
  5. contextual input fields;
  6. approving Decision;
  7. supporting Evidence;
  8. unresolved Finding.

This is enough to support:

---

Acceptance criteria

A master-data lineage implementation should not be considered complete merely because every target table has an upstream table.

For each critical target field, verify:

Identity

Granularity

Transformation

Governance

Evidence

History

When these checks fail, table-level lineage should be labelled as partial rather than complete.

---

Common arguments for stopping at table level

“The ETL tool already shows lineage.”

It shows technical dependencies.

It may not show business meaning, authority, local exceptions or decisions.

“Field lineage is too expensive.”

Full field lineage for every field may be.

Critical-field lineage is far cheaper than repeated migration defects.

“The mapping workbook already contains fields.”

A workbook may contain the mapping but not stable identity, rules, evidence and approved history.

“The SAP table tells us the object.”

One table may contain several semantic contexts, and one business concept may span several tables.

“We can investigate manually when a defect appears.”

That preserves dependency on the original experts and increases cutover risk.

“AI can infer it later.”

AI can propose likely connections. It cannot recover missing approval and authority reliably.

---

What Martenweave should build next

Martenweave already has:

The next vertical slice should introduce a critical-field lineage completeness check.

Goal

Identify SAP target fields that have only dataset- or table-level lineage but lack governed field-level context.

Scope

For selected FieldEndpoints, require:

Example warning

MW-LINEAGE-004

Target:
FEP-S4-KNVV-KDGRP

Table-level upstream:
CRM_CUSTOMER

Missing:
- approved source FieldEndpoint;
- Sales Area context;
- transformation classification;
- supporting Decision.

Acceptance criteria

The Customer Group example must distinguish:

table dependency:
CRM_CUSTOMER → KNVV

governed field lineage:
CRM Segment + Sales Area
→ enrichment Mapping
→ Customer Group
→ KNVV-KDGRP

Validation command

martenweave validate --repo examples/customer_bp_model

Functional verification

martenweave trace ATTR-CUST-SALES-CUSTOMER-GROUP \
  --repo examples/customer_bp_model

martenweave impact FEP-S4-KNVV-KDGRP \
  --repo examples/customer_bp_model

This would strengthen existing functionality without turning Martenweave into a generic pipeline-observability platform.

---

Final perspective

Table-level lineage answers:

Which datasets and systems are connected?

Master-data lineage must also answer:

Which governed value moved, under what meaning, for which business context, through which transformation and with whose approval?

For SAP migration, the useful chain is not merely:

Legacy table
→ staging table
→ SAP table

It is:

source field
+
record keys
+
organisational context
→ transformation
→ business Attribute
→ SAP target field

supported by:

Rule
Decision
Evidence
ownership
effective baseline

Table-level lineage remains valuable.

It is the map of the highways.

Master-data migration fails at the junctions:

Those failures are invisible at table level.

The practical test is:

Can the programme explain why one specific Customer received one specific Customer Group in one specific Sales Area—and trace that result to source fields, transformation logic, a governed Attribute, an approved Decision and test Evidence?

When the answer is yes, the lineage is useful for master data.

When the answer is only:

The customer table feeds KNVV,

the graph is technically correct and operationally incomplete.

About the authors

Martenweave is maintained by Dzmitryi Kharlanau.

Martenweave is a backend-first model-governance and evidence layer for SAP migration, MDM, data governance and AMS teams.

It complements technical dataset lineage by connecting source and target fields to business Attributes, contextual Entities, Mappings, Rules, Decisions and Evidence.

The objective is not to replace pipeline-lineage tools.

It is to capture the model knowledge those tools usually cannot prove.

Sources and notes

This article was reviewed on 14 July 2026.

OpenLineage’s current object model represents lineage through Datasets, Jobs and Runs and supports both runtime observations and design-time metadata. It treats database tables and similar discrete data collections as Datasets connected through transforming Jobs.

OpenLineage’s column-level lineage facet extends dataset dependency with information about which input fields produce output fields and how. It distinguishes direct identity, transformation and aggregation from indirect dependencies such as joins, filters and conditional logic.

Martenweave’s current public example generates lineage edges from canonical model files and provides trace and impact operations connecting business Attributes, physical SAP FieldEndpoints, Mappings, Decisions and Evidence.

Martenweave’s documented pipeline validates canonical files, builds generated indexes, detects gaps, runs lineage and impact analysis and routes proposed changes through human review rather than direct system mutation.

The layered-lineage model and critical-field completeness check proposed in this article describe a recommended Martenweave product direction. They should not be interpreted as a claim that every illustrated lineage object or validation code already exists in the current release.

Martenweave is independent and is not affiliated with or endorsed by SAP or OpenLineage.

Primary sources

Related articles