Classic DWH

The introduction of the core layer as a response to the need to integrate and combine data from different source systems emerges at the Classic data warehouse (DWH) maturity level. This requires a decision about the technology stack that will be used in the long term to support the solution.

This decision goes hand in hand with the selection of the core layer model and the creation of important processes for the core (e.g. historisation principle) and the whole solution (e.g. increased complexity for orchestration).

At this point, the importance of metadata starts to increase as the users struggle to understand the relations in the core layer and the transformation from stage to mart layer when using the final outputs. The creation of the core layer requires the allocation of a BI architect to govern the processes related to data integration and to ensure a proper ownership approach for the core.

What you have

• A vision of what you want to achieve with the solution, which business processes you want to support and what technology stack you want to use
• Independent BI team capable of operating the solution, developing new features and supporting users
• Defined core model and architecture pattern
• Core layer that allows you to combine data from different systems and create a unique source of truth for subsequent usage
• Well-formalised change management process to keep the core layer stable
• Dedicated non-production environments for development and testing
• SLAs you have to respect
• An automated daily transformation of data supported by a well-described ETL process with all needed metadata

What you need

• Strong architecture to manage the creation of the core layer
• A robust Data Quality process that leads to the correction of data in the source systems
• An Reference Data Management solution to manage code tables more efficiently
• An Master Data Management solution to deduplicate clients in order to satisfy the demand for this kind of service
• A tool to anonymise data for testing and development (find out more about Data Masking Tool)