German Manufacturing Conglomerate

The Challenge

The conglomerate operated four distinct business domains (GD, SCP, SXT, and a processing division), each with:

• Different data sources (SAP OData, Siemens Historian, SharePoint, XRD crystallography files)
• Different analytical needs (operational dashboards, quality analytics, supply chain optimization)
• Different compliance requirements

Building four completely independent platforms would have been expensive and unmaintainable. Building one monolithic platform would have forced inappropriate compromises on each domain.

Our Solution

Shared Architecture, Domain-Specific Configuration

We designed a single platform architecture template that could be instantiated per domain with domain-specific configurations:

• Azure Data Factory pipelines parameterized to handle different source systems
• Databricks workspaces per domain with shared libraries and domain-specific notebooks
• Terraform modules that provision consistent infrastructure while allowing domain-level customization

Domain-Specific Integrations

Each domain required specialized data ingestion:

• SharePoint ingestion — Automated extraction of documents and metadata from SharePoint libraries for the administrative domain
• Siemens Historian — Time-series data from industrial control systems for the manufacturing domain
• XRD crystallography — Parsing specialized scientific file formats for the materials research domain
• SAP OData CDC — Change data capture from SAP systems for the supply chain domain

ParserFactory Pattern

To handle the diverse data formats, we built a ParserFactory framework: a base parser class with domain-specific implementations. New data formats can be supported by adding a parser implementation without modifying the core pipeline code.

Streamlit Dashboards

Each domain received Streamlit-based dashboards tailored to their specific analytical needs, deployed alongside the data platform.

Results

• 4 production platforms deployed from a single architecture template
• 70% faster deployment for subsequent domains after the first was built
• Consistent operations — the same monitoring and alerting patterns work across all domains
• Domain autonomy — each domain team manages their data and dashboards independently

Technologies Used

Azure Data Factory, Databricks, Delta Lake, Terraform, Streamlit, SAP OData, Siemens Historian, Python, SharePoint API