Development and implementation of a system to support prediction of suicide risk in the Department of Veterans Affairs - DR. Robert Bossarte and Paul Bradley

1. Development and implementation of a system to support
prediction of suicide risk in the Department of Veterans Affairs
Perceptive Reach
Integrated Reach Database System (IRDS)
Dr. Robert Bossarte / Paul Bradley
1
(PricewaterhouseCoopers PS LLP)(Department of Veterans Affairs)

2. 2014: The Problem
2
The most recent suicide data released by the US
Department of Veterans Affairs (VA) indicates the rate of
Veterans suicide remains largely unchanged over the
past three years. About 22 Veterans take their own life
every day. While studies show decreases in suicide rates
for Veterans who seek care within VA’s health system,
reaching at-risk Veterans and persuading these
individuals to seek care remains a challenge.

3. Perceptive Reach Innovation At a Glance
3
Summary
The Perceptive Reach development and field pilot
combines technology, outreach and clinical
services to realize a data-driven early intervention
and treatment solution aimed at upstream
suicide prevention. PwC developed a solution for
analyzing multiple data sets to identify at-risk
individuals and provide secure notifications of these
results to Veteran support services. The long-term
vision of the Perceptive Reach system is to provide a
platform from which the general health and wellness of
the Veteran population can be improved through data-
driven early intervention solutions.
Schedule
Key Milestones & Deliverables
Phase 1
(9 months)
Sept’14 start
• Reach database
• Data analytics platform
• Reach Dashboard
• Direct messaging
Phase 2
(3 months)
• Internal IT pilot
Phase 3
(Optional Task)
(12 months)
Sept’15 start
• Field pilot
• Outreach and intervention
• Continuous monitoring
• Evaluation and results report
• Executive briefing
Impact
• Early intervention and treatment
• More expedient and effective methods to detect
previously unknown at-risk Veterans
• Establish extendable platform for future innovations
• Standardized methods to interface to new data sources
• Lower cost of ownership via open source solutions
Funded through the VA Center for Innovation

4. 1 2 3
Identification Evaluation Intervention
Perceptive Reach Methodology Overview
The Perceptive Reach
solution uses predictive
analytics to identify at-
risk patients, and sends
the results to providers.
The Perceptive Reach
solution provides tools
for reviewing, managing,
and understanding the
population of at-risk
patients. Tools include
secure notifications, and
dashboards for viewing
individual, facility,
regional and national
data.
Care team members
evaluate the patient using
information in the
Perceptive Reach
Dashboard and the
primary electronic health
record.
Care team members
consult and collaborate
with other providers and
determine options for
intervention. The
Dashboard provides
clinical decision support
to help guide and inform
care decisions.
A care team member
reaches out to the at-risk
patient, and discusses
care options.
Care providers work with
patients to establish
Safety Plans, increase
coping skills, optimize
clinical treatment,
enhance support
networks, and improve
lives.

5. Perceptive Reach Concept Diagram
5
CDW
DATABASE to retrieve
Veterans risk factor data
ANALYTICS to
Identify Veterans
risk levels
SECURE MESSAGING
to notify outreach
& intervention staff
WEB DASHBOARD to
provide analytics and
clinical to support outreach &
Intervention staff
Outreach & Intervention
1
2
3
5
4

6. Perceptive Reach Dashboard
6

7. Perceptive Reach Dashboard
7
SURVEILLANCE VIEW

8. Perceptive Reach Dashboard
8
SURVEILLANCE VIEW

9. Perceptive Reach Dashboard
9
FACILITY VIEW

10. Perceptive Reach Dashboard
10
INDIVIDUAL VIEW

11. Perceptive Reach Data Model
11
VAMC = VA Medical Center
CDW = Corporate Data Warehouse
SPAN = Suicide Prevention Applications Network
VCL = Veterans Crisis Line
SDR = Suicide Data Repository
-M = Mortality Data
NDI = National Death Index
…
CDW
VAMC
VistA
SDR
VAMC
VistA
VAMC
VistA
SPANVCL
DOD-MVA-M
CDC NDI
IRDS

12. Risk Assessment Model Background
12
The goal of the suicide completion risk model is to aid in the identification of
individuals at a heightened risk for suicide in a given month:
• The model was originally developed by the Serious Mental Illness
Treatment Research and Evaluation Center (SMITREC) team at the VA
• Logistic regression models of suicide risk among VHA patients
• Suicide completion was predicted using a logistic regression model with
clinically meaningful input variables
• The predicted suicide completion probability is translated into outreach
recommendations:
• Top 0.1% of predicted probabilities: Individual-specific clinical outreach
• Top 5% of predicted probabilities: Population health programs and
interventions

13. Model Inputs
13
Data inputs include clinical and demographic data:
• There are 381 variables (including interactions) that are used in the risk
model which are categorized as follows:
• Medical diagnoses
• Prescriptions
• Medical procedures
• VHA usage
• Military sexual trauma
• Clinic stop codes
• Veteran status
• Demographics
• Modelled in SAS and R
• Implemented in SQL

14. Perceptive Reach Component Diagram
14
Enterprise Operations Server
@AITC
CDW
VACI_IRDS
Project DB
CDW DBs
Windows Server OS
MS SQL
Server
Node.js,
Express.js,
Angular.js,
Jqueury,
Reach DB
Dashboard App
Direct
Msg App
IRDS Data Source
IRDS Data
Staging
Area
IRDS Application
VLER
Direct
Service
CONFIGURATION
Secure
Messaging
CDW
RESTful
SSIS

15. Open Source Software
15
Node.js
AngularJS
Express
DataTorrent (Dashboard)
HTML, JavaScript, CSS
socket.io
DataTorrent (Dashboard) - Generic
Open Source AngularJS dashboard
leveraging widget driven functionality.
AngularJS - built/maintained by Google,
Extends HTML, and RESTful API
Friendly.
Express - Thin layer web application
framework on top of Node.js that
simplifies and streamlines HTTP and
RESTful API.
Node.js - lightweight and efficient, perfect
for data-intensive real-time applications.SQL Server
SSIS Imports
Stack Highlights

16. Open Source Tools
16
Bower.io
(Package Manager)
(JS Test Runner)
(JS Task Runner)
(Node.js Package Manager)
(Continuous Integration Server)
(Behavior Driven Testing)
(Collaboration & Tasking)
(Socializing Ideas)
(Code Repository)
Continuous Integration

17. Perceptive Reach Usage Model
17
CDW
Perceptive Reach / IRDS
Reach DB
Secure
Messaging
App
Dashboard
App
VLER Direct
Messaging
Service
Web
browser
Outreach
& Intervention
Team
On a nightly basis, retrieve
updated records from CDW to
form new population to run
through the surveillance model

18. Perceptive Reach Usage Model
18
CDW
Perceptive Reach / IRDS
Reach DB
Secure
Messaging
App
Dashboard
App
VLER Direct
Messaging
Service
Web
browser
Outreach
& Intervention
Team
Run the surveillance model and
compute risk scores for new
population. Identify Veterans in
the “Top” and “Middle”
stratification levels.

19. Perceptive Reach Usage Model
19
CDW
Perceptive Reach / IRDS
Reach DB
Secure
Messaging
App
Dashboard
App
VLER Direct
Messaging
Service
Web
browser
Outreach
& Intervention
Team
Assemble secure messages
listing the Veterans in the “Top”
and “Middle” stratification levels.
Transmit message through the
VLER Direct service.

20. Perceptive Reach Usage Model
20
CDW
Perceptive Reach / IRDS
Reach DB
Secure
Messaging
App
Dashboard
App
VLER Direct
Messaging
Service
Web
browser
Outreach
& Intervention
Team
User receives message
notification and retrieves secure
message from VLER Direct
service

21. Perceptive Reach Usage Model
21
CDW
Perceptive Reach / IRDS
Reach DB
Secure
Messaging
App
Dashboard
App
VLER Direct
Messaging
Service
Web
browser
Outreach
& Intervention
Team
Using the link in the secure message, the user
navigates to the dashboard, and signs-in to
see additional details regarding Veterans

22. Next Steps
22
CDW
1. Pilot solution at two VAMCs between February and September 2016
2. Enhance prediction model – reduce variables from 381 to <50
3. Enhance dashboard
A. “Move the needle”
B. Highlight need for new prediction models
C. Highlight opportunities to address additional problems with this approach

23. Questions?
23