Chesapeake Regional Information System for our Patients (CRISP) is a nonprofit healthcare information exchange (HIE) whose customers include states like Maryland and healthcare providers such as Johns Hopkins. CRISP’s work supports the local healthcare community by securely sharing the kind of data that facilitates care and improves health outcomes. When the pandemic started, the Maryland Department of Health reached out to CRISP with a request: Get us the demographic data we need to track COVID-19 and proactively support our communities. As a result, CRISP employees spent long hours attempting to handle multiple data sources with complex data enrichment processes. To automate these requests, CRISP partnered with Slalom to build a data platform powered by Databricks and Delta Lake. Using the power of the Databricks Lakehouse platform and the flexibility of Delta Lake, Slalom helped CRISP provide the Maryland Department of Health with near real-time reporting of key COVID-19 measures. With this information, Maryland has been able to track the path of the pandemic, target the locations of new testing sites, and ultimately improve access for vulnerable communities. The work did not stop there—once CRISP’s customers saw the value of the platform, more requests starting coming in. Now, nearly one year since the platform was created, CRISP has processed billons of records from hundreds of data sources in an effort to combat the pandemic. Notable outcomes from the work include hourly contact tracing with data already cross-referenced for individual risk factors, automated reporting on COVID-19 hospitalizations, real-time ICU capacity reporting for EMTs, tracking of COVID-19 patterns in student populations, tracking of the vaccination campaign, connecting Maryland MCOs to vulnerable people who need to be prioritized for the vaccine, and analysis of the impact of COVID-19 on pregnancies.

1. Unlocking the Power of
Data During a Pandemic
Steve Dowling & Andy Hanks
From Vaccine Management to ICU Planning: How CRISP &
Slalom Unlocked the Power of Data During a Pandemic

2. Agenda
§ The Need for a Data
Lakehouse in Healthcare
§ The Power of a Lakehouse
§ Platform Architecture
§ Metadata-driven Pipelines
§ Using Data To Fight COVID-19

3. The Need for a Data Lakehouse in Healthcare

4. Solving Healthcare’s Data Problem
Why CRISP chose to build a Data Lakehouse in the middle of a pandemic
• Data Everywhere!
• Healthcare has one of the most complex data ecosystems with data coming from providers,
health departments, research institutions and more
• Lack of Data Standardization
• With data coming from so many different sources the medium used to transfer and store data
varies greatly from database tables, csv files, json logs, etc.
• The Need for Quick Answers
• The pandemic escalated a need for health care providers to use their data to solve complex
questions such as “Which patients are at the highest risk for severe disease”

5. The Power of a Lakehouse

6. The Power of a Lakehouse
Csv
Cobol w/
Copybooks
Compressed
Csv (zip, bz2)
Cobol FWF
Json Parquet
~ 400 GBs
per day
Data Volume
Sources Ingested
Data
Transformation
Data Blending
Data Deduplication
Data Enrichment
Capabilities
Data Visualization
Location Analysis
Automated Email Reports

7. Platform Architecture

8. Platform Architecture Overview

9. Metadata-driven Pipelines

10. Metadata-driven Pipelines
Example Code Snippet:
Why use metadata-driven pipelines?
Faster Set-up
Reusability
Built In
Documentation
Standardized
Process

11. Using Data To Fight COVID-19

12. Using Data To Fight COVID-19
How CRISP utilized data to help in the battle against COVID-19
COVID Testing Analysis
Analysis of High-Risk Groups
Vaccination Planning
COVID Analysis for Universities
ICU Capacity Reporting
Vaccination Reporting
High Impact Research Studies
COVID Hospitalization Analysis
Near Real Time Contact Tracing

13. Using Data To Fight COVID-19 Cont.
Using Databricks CRISP was able to do the following*
>10M
COVID-19 Tests
Processed
3.5M
COVID-19
Vaccinations
Processed
7.5M
Vaccine
Registrations
Processed
5.5M
Addresses
Geocoded
*Data current as of April 13th, 2021

14. Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.