Case Study
Automated Solution Accelerates Data Validation To Improve Efficiency
- Customer: Multi-state healthcare organization
- Challenge: Import a large volume of data from a legacy lab system into Epic’s Beaker application
Results
- 50,000 records validated automatically in only eight weeks
- 99.7% data validation matching
- 520 working days of manual time saved
- $150,000-$250,000 labor cost savings
Definitely saved us a bunch of money (and) helped us identify a big gap in the data.
CLIENT
Background and Challenge
A leading multi-state healthcare organization was converting and importing a large volume of data from a legacy lab system into Epic’s Beaker application. Before importing this data into the Production environment, they needed to confirm the imported data matched the exported data and identify any potential errors. Doctors use the imported data to treat patients and monitor their progress. Altered lab results pose a risk to a patient’s well-being, so the migration would be canceled if they could not ensure a near-perfect matching of the imported and exported data.
The traditional, manual method for this effort is time intensive, error-prone, and costly. They would need 40-plus temporary chart reviewers to review the data for errors. The organization sought a cost-effective solution that would reduce effort, promote accuracy, save time, and be repeatable for future validation projects.
Solution
Tegria experts developed a plan to take a sampling of data from the legacy lab system, import that data into a copy of the Production system, and then use a raw, pre-imported dataset to perform an automated validation. The format would display results in a sortable format with a separate error log for corrections. Automation would help remove human error, increase productivity, and provide an easily repeatable process.
The organization provided the raw legacy lab files as the source of truth and access to their Dev Clarity Beaker environment, which housed the data needed to compare. The Tegria team built a standalone SQL database within the client’s infrastructure and configured the appropriate database connections on a virtual desktop to perform this work remotely. The internal IT team installed the required tools on the virtual desktop, including Visual Studio, SSMS (used to deploy the SSIS package), Oracle SQL Developer, and Notepad++.
These tools allowed Tegria’s data architect and engineer to query and extract data from Clarity, import data into the standalone SQL database, code and run the automated validation job, and export results for review. Tegria met with the client’s lab conversion team after each round of validation to determine the validity of errors, help identify root causes, and receive feedback to make necessary changes for the next round of validation.
The first round of validation on 10,000 cases was performed with a matching percentage rate of 89%. The second round of validation on 20,000 cases had a matching percentage rate of 99.74%. A third and final round of validation on 20,000 cases resulted in a matching percentage rate of 99.76%.
In addition to delivering the automated solution, Tegria provided two training sessions and instructions for running the automated process. The training included making changes to the SSIS package and configuration files to use the solution for other Epic and non-Epic validation efforts. Using this solution for any future validation effort will produce long-term cost savings.
Results
The automated solution compared 50,000 records (277,763 rows of data) over eight weeks. In contrast, the manual method was estimated to take five minutes per record for a total of 4,166 working hours (~520 days) to review all 50,000 records.
With a data validation matching percentage of 99.7%, the automated solution gave the organization complete confidence that the legacy data migrated into the Epic Beaker Production environment was of the highest quality. The zero to 0.3% with discrepancies were sent to an error log for correction before being imported to ensure that no errors fell through the cracks.
The solution decreased the workload by providing an automated system and repeatable process. The estimated initial labor cost savings were between $150,000 and $250,000. By applying the solution to other projects, the potential savings will grow exponentially.