In this age of “Big Data”, and even bigger plans for how companies are planning to use their data to gain a competitive advantage, one pre-Big Data era concept still remains relevant – those data warehouses are not of any value if the data cannot be trusted.
Data Quality is the true backbone of any data warehousing solution, and that term can be split into 2 simple concepts: 1) understanding what you have, and 2) comparing what you have to what you want. In technical terms, we call #1, “Data Profiling” and #2, “Data Auditing”.
At Making Data Meaningful, we have designed a successful Data Profiling process that enables us to extract the underlying source table structures within seconds and share that information in a meaningful way with Business Users. Once we have completed that initial assessment, we then work with the Business Users to determine the business rules that will become the basis for the Data Auditing processes.
In order to make it easier for the Business Users to participate in the Data Auditing process, Making Data Meaningful has designed a system that uses Error Handling tables to capture data that does not conform to the business rules they have identified. The data in those tables are exposed to the Business Users using user-friendly BI tools like Tableau, MicroStrategy, Business Objects, etc. or even Excel so that these records can be viewed by the Business in a format that is simple to understand.
Lastly, Making Data Meaningful understands that data quality is an ongoing process so we work with our clients to ensure that they have a Business Intelligence Competency Center (BICC) in place so that Data Governance remains one of the top tasks within the data warehouse solution even after we’re gone. We help organizations define the role of Data Stewards and assist those Stewards with the Data Auditing process by working with them to generate a plan of action to either correct non-compliant data or modify the business rule.
Although Data Profiling and Data Auditing may not get the same glamorous attention that Big Data does, they are terms that are critical to the success of any data warehousing solution. They are processes that Making Data Meaningful has invested a lot of effort into refining for the benefit of our clients.
The post What Good is a Data Warehouse If I Can’t Trust the Data? appeared first on Making Data Meaningful.
