.svg)
A Data Product Health Score is only as complete as the lineage underneath it. Enterprise pipelines move data from streaming systems through transformation layers into open table formats and analytical warehouses, often across cloud boundaries. ADOC 26.6.0 extends Asset Correlation to four platforms common in these architectures: Amazon Redshift, AWS Glue, Google Cloud Pub/Sub, and Apache Iceberg. For teams already using OpenLineage, the experience is identical; the coverage simply reaches further across the hybrid estate.
How OpenLineage Coverage Works
OpenLineage extended support connects pipeline lineage to catalog assets. When a pipeline runs, it emits OpenLineage events describing the datasets involved. ADOC processes those events, creates dataset nodes in the pipeline graph, and matches each node to a previously crawled catalog asset using the dataset namespace, name, and available metadata facets. When a match is found, the pipeline node links directly to the catalog asset — navigable from both the pipeline and asset views.
Key distinction: No changes are required to how pipelines emit lineage. Matching happens automatically on the ADOC side as long as the catalog asset has been crawled and the OpenLineage event contains a resolvable dataset identifier.
Every platform added here is another source the Data Product can see, score, and trust. For agentic AI systems drawing from hybrid estates, broader lineage coverage is what makes the product's runtime trust signal actually reliable.
What’s New in 26.6.0
Amazon Redshift
Redshift datasets are identified by cluster endpoint, region, and port in the namespace, with database, schema, and table in the name field. Pipelines that read from or write to Redshift — including S3 export jobs — now correlate to the corresponding catalog assets, making Redshift a fully connected node in multi-stage lineage.
AWS Glue
Glue jobs frequently read from Glue Data Catalog tables while writing to S3, which means the physical storage path and catalog identity differ. ADOC resolves this by processing symlink facets emitted by Glue jobs — the authoritative reference to the underlying catalog table — rather than treating the S3 path as the dataset.
Google Cloud Pub/Sub
Pub/Sub datasets are identified by project and topic or subscription identifier. When a pipeline publishes to or consumes from a Pub/Sub topic, ADOC correlates that event to the corresponding catalog asset, bringing streaming ingestion steps into the same lineage graph as downstream transformation and storage layers.
Apache Iceberg
Iceberg support covers two catalog configurations: REST catalog and Iceberg tables registered in AWS Glue. REST catalog environments use catalog facets from the OpenLineage event; Glue-registered Iceberg tables use the same symlink and catalog facet handling as Glue. Both input and output tables are correlated.
What This Looks Like in Practice
A few scenarios that become traceable with this release:
- Streaming analytics: A business event published to Pub/Sub, processed through Glue, stored in Iceberg, and loaded into Redshift, now linked as a single connected lineage view across all four stages, and fully visible to any Data Product composed from those assets.
- Warehouse lineage: A Redshift reporting table sourced from Iceberg datasets. Teams can navigate upstream through correlated lineage to find where a data quality issue originated, without manual cross-system investigation.
- ETL validation: Data engineers can verify that catalog assets produced by a Glue job match expected downstream assets, catching discrepancies before they reach consumers.
Who Benefits
Most relevant for teams that operate hybrid or multi-cloud pipelines, combine streaming and batch architectures, or need faster root cause analysis when data quality issues surface in downstream reports:
- Pipelines spanning AWS and GCP — Redshift and Glue on one side, Pub/Sub on the other
- Lakehouse architectures using Iceberg as the intermediary layer between ingestion and analytics
- Data quality investigations that currently require manual cross-system tracing to find upstream sources
- Data Product owners whose Health Scores depend on lineage completeness across a hybrid estate — and agentic AI systems that need to know every upstream source is accounted for before acting on a product.
Getting Started
OpenLineage coverage is infrastructure for Data Product trust. The broader the platform coverage, the more complete the runtime signal the product carries.
Support for these platforms works the same way as for existing sources. To enable it:
- Onboard and crawl catalog assemblies for the relevant platforms (Redshift, Glue, Pub/Sub, and/or Iceberg) in ADOC
- Ensure pipeline lineage is being emitted via OpenLineage for input and output datasets on those platforms
- Open a pipeline run or catalog asset — correlated assets appear automatically once discovery and lineage are both available
For emitter formats, namespace matching rules, symlink and catalog facet requirements, and troubleshooting guidance, see the OpenLineage asset extraction and correlation guide in the documentation portal.
