As we first introduced within the first part of the Data Vault Use Cases article series, the Enterprise Data Warehouse (EDW) can do more than just simple reporting and dashboarding.
We previously explored how the EDW can help to improve data quality by implementing data cleansing rules.
This can be applied by write-back operations that affect the source systems directly. Though this was only one example of how to add more value to the EDW.
The scalability and flexibility of Data Vault 2.0 offers a whole variety of use cases that can be realized, e.g. to optimize and automate operational processes, predict the future, push data back to operational systems as a new input or trigger events outside the data warehouse, to name a few.
Figure 1: Data Vault Use Cases
AN EXTERNAL INTERFACE FOR BUSINESS PROCESS AUTOMATION
The next use case to be introduced in the series is an overall concept of business process automation. The EDW can be used as the central storage not only for data cleansing rules but also for a business process automation framework.
Typically, when a business is running, content in operational systems will be created, deleted or updated. As soon as these operations arrive in the EDW, an automated process can act upon it. This can either happen by business process automation frameworks, an Enterprise Service Bus in a Service-Oriented Architecture, or an ordinary shell script, to name just a few.
To filter for specific events, business rules need to be defined and implemented as Business Vault entities. To initialize the business process application, it needs information in a specific structure, namely those contain the trigger criteria.
For this, the information marts can be used as interfaces, drawing upon their first appearance as we already have called them interface marts in part 1 of our series. Using virtualized Tables (Views) is a good practice in this case as one does not need to move data again and changes can be applied very quickly.
One example is to use new entries in a view as a trigger event, followed by an action in operational systems, the EDW itself or other third party systems.
Here is an example of how we can use such a process to feed our knowledge management system (KMS) with combined information from different sources:
Our operational systems act as short-term memory of the organization with the latest data available while the EDW acts as the long-term memory, storing all cleansed and history data. When the integration into the EDW is done, virtualized tables in the Business Vault apply business rules to cleanse and enrich KPIs to the data. The interface mart, also virtualized, selects data from the Business Vault by applying a filter criteria.
The final result is what is the desired set to be forwarded to the KMS automatically. The following image provides an overview regarding the architecture:
The Knowledge Management System (KMS) is used to store thousands of pages with business processes, development documentation, project experience reports, training materials, Data Vault knowledge, and quite a bit more.
To create and maintain this knowledge, a significant amount of manual work is required. This is the reasoning behind automating the process as much as possible.
Therefore, we apply the business process automation to documentation work:
As soon as specific objects enter a predefined status in one of our source systems, it is caught by the Interface Mart where the business automation framework is waiting for new records. It then acts on newly created record events and creates a new page with a pattern based SQL-Statement which is used by a database plug-in in the KMS.
For security reasons we created a special user account, which has only access to the KMS interface mart and does not have access to other areas in the data warehouse.
In use cases as such, the EDW provides great additional value by adding documentation pages which are automatically maintained, which in turn saves a significant amount of working hours.
Get Updates and Support
Please send inquiries and feature requests to [email protected].
To support the creation of Visual Data Vault drawings in Microsoft Visio, a stencil is implemented that can be used to draw Data Vault models. The stencil is available at www.visualdatavault.com.