Key decision makers in every business should understand the value of data and the value this data has when it comes to making evidence-based decisions. With data, you can predict the likely outcome and the chances of success or failure of a decision. Where decisions involve significant investment or affect large numbers of people, data does a lot of heavy lifting. 

Increasingly, businesses describe themselves as “data-driven” and see data as a core business asset. But data that is poorly collected and curated, misused or not held securely has limited value. 

When you think about business data, it is helpful to think about its lifecycle. This starts when the data is collected and stored. It progresses to being managed and shared by users, and ultimately is used to automate processes. In the earliest stages of data collection and storage, this provides the foundation for the entire lifecycle. That is why it is so important to ensure the strength of your data platforms and the integrity and reliability of your data. Missing or incomplete data at this stage (the first of the three pillars in Insight’s data lifecycle strategy) puts decision-making at risk and reduces the trust that decision-makers have in the data. So, they are back at square one – making decisions based on instinct. 

Robust data collection and storage is essential for the integrity and reliability of data, but what do we mean by this? 

 

You need data you can trust 

Data reliability is easy to grasp. If data is unreliable, people won’t use it. That is true even if it is only perceived to be unreliable. Decisions based on unreliable data will be challenged and rightly so! In today’s data-prolific world, it is hard to justify the intelligence behind a decision where the data isn’t considered reliable.   

This is why data integrity plays such an important role. Data integrity is the continuous assurance of the overall accuracy, completeness and consistency of data, throughout the lifecycle. 

Data integrity isn’t a given. There are many ways in which data integrity can be compromised. For example, where data is incorrectly formatted or stored in the wrong structure. Where back-up and recovery processes are unstable or out of synch with business needs. Where there are gaps, duplicates or dummy entries. And where data is exposed to security threats or is corrupted. Processes and structures need to be in place that can protect data from these compromises. 

 

Trusting the source and the journey  

Data integrity also means being able to track the data back to its original source and have confidence in its provenance. This is essential to comply with strict privacy regulations. It also ensures that data is only used if it comes from reliable and legitimate sources. 

Integrity and reliability are critical to the quality of business data. If you don’t know where the data has come from or you do not trust its accuracy and/or the completeness, it’s hard to believe in its quality. In a long-running survey of business intelligence and analytics users, 40% said data quality issues were the second most common barrier to data-driven decision making. Given that 62% of companies in the same survey want to treat information as an asset, organisations clearly need to protect the reliability and integrity of their data asset. 

 

Data Integrity and Reliability in the Real World 

Just how do concepts like data integrity and reliability translate to the real-world? We recently helped a global engineering business that was facing a number of data integrity and reliability challenges. The business collects and analyses data from various sources, including its customer relationship management (CRM) systems, and its Enterprise Resource Planning (ERP) systems, in particular its HR and Finance departments. 

Different departments including finance, procurement and operations, were each producing their own sets of figures, and the data was being held in various locations around the globe. People questioned the validity of decisions made on such disparate datasets. For effective, trusted decision-making, the business wanted to create a centralised source of reliable data insights that everyone could agree on. The data needed to be cleaned and consolidated into one integrated view so it could be used for accurate reporting and analysis. 

As well as building a data warehouse hosted in the engineering firm’s Azure cloud environment, insight provided guidance on how to move data into the new environment and trained staff in generating accurate reports from reliable data. The whole project was delivered within one month of the start date.