FDA News Webinar - Inspection Intelligence

1Copyright © 2016 Qualified Data Systems Inc.
Inspection Intelligence
Developing a Data-Driven Approach
FDA News Webinar
December 19th, 2016
By: Armin Torres, Principal Consultant – Qualified Data Systems

Inspection Intelligence Overview
Inspection Intelligence– Knowledge acquired through the collection, aggregation,
analysis, and interpretation of information sources which enable proactive
compliance readiness.
The Data-Driven Digital Quality Approach:
• Uses External and Internal data sources to develop a complete 360 degree view of your
Compliance Readiness status
• External Data sources highlight your organization’s Quality and Compliance Risk Profiles using
post-market feedback channels
• Internal Data sources are intended to prepare your organization for inspections using quality and
compliance metrics derived from key QMS systems. This is your Vital Signs Monitor.
• Five key Medical Device QMS Systems for Vital Signs Monitoring include:
• Corrective & Preventive Action (CAPA), Production & Process Controls (P&PC), Management Controls (MGMT),
Design Controls (DES), and Document Controls (DOC)

Intelligence Information Sources
Industry
Providers
Pharma, Device, Biologics, Cosmetics,
other Industry Groups
(Product Life Cycle Data)
Academic, Independent,
Consulting Studies,
Medical Literature,
Publications
Hospitals, Clinics, Diagnostic Centers
(Event Data)
EU, APAC
USA, LATAM
ROW
(Regulations)
Insurance Companies
(Claims Data)
HDO’s
Sources of Potential
Emerging Signals
Regulatory
Intelligence
User Experience
Reporting
(Sentiment Data)

Intelligence Platform
External Data Sources
Data Remix & Mining
Internal Data Sources
DATA
WAREHOUSE
REGULATORY
INTELLIGENCE
REPORTING
ALERTS STATISTICAL
VISUALIZATION
ETL PROCESS
EXTRACT
VALIDATE CLEAN
LOAD
QMS Supply Chain Manufacturing Customer Service
CAPAAdverse Event Deviations Risk Mgt. Development Maintenance
Public Domain Data (API)
Proprietary Data - Contextualization
Products & Services
Regulatory Science Innovation
Pre-Production, Production, & Distribution Data
Post-Market Crowdsourced Data

Intelligence Platform
• Provides a holistic view of external environmental factors and internal
performance data which can be measured, baselined, reported, and prioritized for
improvement.
• Can help identify areas of regulatory overlap and gaps
• Assists in prioritizing Risk-based reviews based upon key performance indicators
• Facilitates Product & Service Quality improvements through the integration (i.e.
“Remix”) of internal and external information sources
• Establishes ownership and accountability for meeting Quality Objectives and
Compliance requirements.
• Establishes a solid foundation for Governance, Risk, and Compliance
Management across the enterprise.

External Open Data?
Open Data Definition – “Open means anyone can freely access, use,
modify, and share for any purpose (subject, at most, to requirements
that preserve provenance and openness).”
In other words:
“Open data and content can be freely used, modified, and shared by anyone for
any purpose”
Why is Open Data so important?
Because licenses and dedications are readily available such as Creative
Commons, Open Data Common Public Domain Dedication, Open Data Commons
Open Database License (ODbL), etc.
Source: http://opendatacommons.org/

External Data Brokers & Services
Use of External Data Brokers:
• Typically offer services for a fee (fixed or A-la-Cart)
• Services often include Searching, Filtering, aggregating, and
limited downloading or reporting facilities
• Payment for these services does not necessarily translate
into higher quality data. Choose wisely!
• Beware of Service Providers who repackage Open Data for a
fee without proper disclosures or aggregate copyrighted
information from the Web.
• Understand that there is an entire industry dedicated to filling
Freedom of Information Requests for undisclosed recipients
(FOIA.gov)
• Exclusions to the FOIA include some Law Enforcement and
National Security Records.
Look for trusted and
validated data vendors
who provide
transparency and
promote open
standards for
information sharing.

Using External Open Data
Benefits:
• Large historical datasets readily available for data mining. Use these as lagging indicators of
performance.
• Self-Service Data discovery, Analysis, and Visualization tools for Lay-Users maturing rapidly.
Vendor product eco-system is robust.
• ROI easier to demonstrate through Proof-of-Concept use cases
• Can provide competitive advantage in some use cases where product performance can be
benchmarked.
Challenges:
• Not all structured data sources use a common standards-based framework
• Often requires specialized IT skills and costly tools to prepare the data for use
• Use case for real-time Advanced Safety Signal Detection is still a long way off but identifying
Emerging Signals may be possible.

Identifying Emerging Signals in Medical Devices
• FDA intends to use on-market performance data to continuously assess device Benefit/Risk Profiles to
identify potential Emerging Signals and communicate them to the public.
• Emerging Signals may represent a new potentially causal association or a new aspect of a known association
between a medical device and an adverse event or set of adverse events.
• Information about device-related signals may come to the attention of FDA through a variety of sources
including, but not limited to, Medical Device Reports (MDRs), MedSun Network reports, data from mandated
post-market studies, clinical trials or data published in the scientific literature, epidemiological research
including evaluation of administrative databases, health care claims data or registries, and inquiries or
investigations from global, federal or state health agencies. In other words, all intelligence data sources.
• Following identification of a signal, the signal first undergoes refinement, whereby additional information from
other data sources is identified, gathered, and evaluated. The signal refinement phase includes interactions
with the device manufacturer(s), unless time does not permit because of the risk of patient harm or it is not
feasible, e.g., CDRH cannot reach all manufacturers.
• Be ready if the FDA calls to discuss a potential emerging signal by
proactively assessing External OPEN FDA DATA!

Digital Factories and their Data
Proactive assessment of Product Safety Signals
• Location where Product Quality is crafted
• Access to data is controlled by internal policies and procedures not external entities
• Timeliness and proximity to data means higher quality, integrity, and value
• Where the product knowledge lives
• Promotes safety innovation versus costly post-market design changes
Internal Data Challenges
• Regulated Data scattered across the corporate environment and often insulated from rest of company
• Data is housed in vertical silos of vendor applications with little interoperability without additional pay walls
being traversed.
• Organizations lacking clarity to enterprise Data Governance, Risk, and Compliance practices
• Data Management maturity highly dependent on company size and financial resources available
• Data Access many times is limited to a select few who posses licenses
• Heavy IT involvement still needed in some areas where skills or tools are not readily available for data
analysis

Data Quality matters
• Recent focus on Quality Metric Programs developed for the Pharma and the
Medical Device Industries in cooperation with FDA, ISPE, and Xavier Health
highlights the need for a renewed emphasis on Data Integrity and Quality.
• While much of the Metric programs (PQLI-Quality Metrics and CfQ-Case for
Quality) focus on metric definition and reporting schemes, many industry
challenges remain to successful implementation. Examples include:
• Making Raw Data available using the least burdensome approach
• Cost of manual or automated data collection, review, and aggregation
• Process development for Digital data extraction, cleansing, integration, and validation prior to reporting
• Training and availability of resources for Data Analysis, Interpretation, and Visualization
• Implementation of Tools for Analytical processing
• Additional Software Validation requirements
• Timely Management Reporting
• Making data actionable in order to operationalize a data-driven culture

Raw Data – Least Burdensome Approach
•Today, Digitization is the road that leads to the least burdensome
approach for collecting, aggregating, analyzing, and interpreting raw
data generated from your Quality Management Systems.
•Digitization generally speaking is the process of converting raw data
into a digital format. This process may include preparing text,
images, audio, video, and numerical information for subsequent use.
•The key advantages to Digitization include:
• Ability preserve, access, and share the data
• Ability to aggregate data from disparate data sources
• Ability to search the digital records
• Ability to analyze, interpret, and communicate the information

Optimizing Costs of Digitization
There are several key aspects of Digitization that can help your organization optimize the costs associated
with data collection, aggregation, analysis, interpretation, and reporting of Intelligence Data:
• Standardization – Standardization can be used to generate a documented data taxonomy which includes
governance and management of your digital assets. The output of standardization should be an SOP that
clearly describes the organization’s data management processes. Additional standardization should be
sought for Technology components used to support your efforts. For example, standardization on Excel,
Minitab, SharePoint, Microsoft BI, SAS, Tableau, or Yellowfin tools and training for personnel should be
considered.
• Consolidation – Too many meaningless metrics and data expeditions can have a severe impact on the
bottom line. A good way to remember this is with the following saying “Not everything that counts can be
counted, and not everything that can be counted counts“. Chose a small set of actionable metrics that
are impactful to your business and get management support before even starting. A useful online reference
to help you get started is provided by Xavier Health. For Medical Devices: http://xavierhealth.org/device-
measures-initiative/ For Pharma: http://xavierhealth.org/pharma-quality-metrics-initiative/
• Process Efficiency – Defining an strategic organizational model to support your data assets is prudent in
saving time, effort, and cost. Different skills sets may be required such as Data discovery, Integration,
Aggregation, Cleansing, Visualization, Analysis, and Reporting. Roles and Responsibilities should be
defined in your data governance SOP’s or Work Instructions.

ETL Basics
Developing a process for digital data Extraction, Transformation, and Loading (ETL) need not be a
difficult or complex endeavor for your External Open FDA Data. Some key considerations include:
• Defining the approach to collect the data (the Extract).
• If you desire to connect directly with OPEN FDA data sources, then you will need to understand how the FDA’s OPEN
DATA initiative works and how you can connect directly using an open-source Application Programming Interface
(API). For more detailed information visit: https://open.fda.gov/
• If you would like a more basic and targeted level of data collection use the FDA databases directly as follows:
• For Medical Devices: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/Databases/default.htm
• For Pharma: http://www.fda.gov/Drugs/InformationOnDrugs/default.htm
• Transforming the Data may involve elements of data validation and cleansing because the OPEN
FDA data is continuous evolving, being enhanced, and changing. Therefore, it may be required to
verify the data schemas and records from time to time. Typically, when something in the database
schemas change your records downloaded will reflect these changes and adjustments will need to
be made on your side.
• The loading process typically involves the storage of the downloaded database records in some
form of consolidated data warehouse where other users in your company can access the data for
subsequent review, analysis, and reporting.

Organizational Needs
Supporting a Digital Quality Metrics Program requires consideration for additional
Training and Development of resources. Some of the top challenges experienced
by companies implementing successful programs include:
• Insufficient Resources;
• Lack of knowledge; and
• The ability to analyze, interpret, and communicate results
Many times resources don’t have the necessary data visualization or statistical
backgrounds required to demonstrate a return on investment to Sr. Management.
This is especially acute when the organization is ready to transition from
descriptive to predictive analytics.

Tools for Analytical Processing
Industry and Vendor solutions have matured in recent years to support the growing need for
business intelligence tools. A good tool to review some of the leading solution providers at a glance
is using the Gartner Magic Quadrant. For further details refer to:
https://www.gartner.com/doc/2668318/magic-quadrant-business-intelligence-analytics
Some of the popular tools used in the Life Sciences include:
• Microsoft Power BI
• Tableau
• Provalis Research
• Yellowfin
• Birst
• SAS
• Qlik
• SAP
• Minitab

Software Validation Requirements
• Begins with robust Data Hygiene
• Good documentation practice procedures
• Data Entry field level verification for automated systems
• Integrity and Business Logic verification for e-Records
• Defined Data Management practices
• 21 CFR Part 11 compliance (Security, Audit Trails, e-Signatures)
• Comprehensive Software Validation Life Cycle for electronic Quality Systems
• Needs to Extend throughout the entire Quality Management System
• To ensure Confidentiality, Integrity, and Availability of Regulated Information Systems
• To promote cross-platform data integration, aggregation, and visualization
• To enable efficient management reporting
• To flatten silos of data for better collaboration, analysis, and problem resolution

The State of Data Driven Decision-Making
• Technology has made Data Analysis more accessible, but:
• Today’s organizations still struggle with the basics of getting their hands around the data needed
to make better decisions.
• Excel Spreadsheets and PowerPoint are still the main entry points for data collection, analysis,
and presentation of descriptive historical information.
• It still takes way too long to aggregate data and contextualize for Sr. Management review
(some take weeks to compile PowerPoints with hundreds of metrics which are tracked
across the business). As a result, many reviews only performed quarterly.
• Near Real-time data review is still a utopian concept.
• Focus from the enterprise continues to be on structured Data Analytics. Show me the numbers!
• Organizations still have little demonstrated capabilities when it comes to managing and making
use of unstructured data (e.g. Text Analytics). 80% of the world’s data is of this kind.
• Data Analytics and self-service enable better data discovery but not necessarily delivers on better
business outcomes.

Making Data Actionable
Select metrics that make the most business sense (gives value) for your organization. The metrics
should be accurate, timely, and contextualized for it to add value in decision making.
For Quality and Compliance consider the following FDA Proposed Metrics:
Drugs:
• Lot Acceptance Rate (LAR) as an indicator of manufacturing process performance. LAR = the
number of accepted lots in a timeframe divided by the number of lots started by the same covered
establishment in the current reporting timeframe.
• Product Quality Complaint Rate (PQCR) as an indicator of patient or customer feedback. PQCR
= the number of product quality complaints received for the product divided by the total number of
dosage units distributed in the current reporting timeframe.
• Invalidated Out-of-Specification (OOS) Rate (IOOSR) as an indicator of the operation of a
laboratory. IOOSR = the number of OOS test results for lot release and long-term stability testing
invalidated by the covered establishment due to an aberration of the measurement process
divided by the total number of lot release and long-term stability OOS test results in the current
reporting timeframe

Making Data Actionable
Devices:
• Pre-Market
• Design Robustness Indicator - Identification of design elements that eliminate, reduce, and prevent
design failures throughout the product lifecycle. A quality design should require minimal changes.
Changes should be enhancements or due to technological advancement.
• Metric: (Total # of Product Changes/Total # of Products with initial sales in the period)
• Production
• Right First Time Indicator - Tracking and trending production-related right first time data to eliminate,
reduce, and prevent repeat failures. First Time Yield metric.
• Metric: (# of units mfg. w/o non-conformances/# of units started)
• Post-Market
• Post-Market Indicator - Analysis of key post market surveillance data to eliminate, reduce and prevent
on-market failures.
• Metric: Service Records * (0.10) + Installation Failures * (0.20) + Complaints * (0.20) + MDRs * (0.20)
+ Recalls (units) * (0.20) + Recalls (total) * (0.10)

Top Challenges in Incorporating Data Analytics
Access to Data
Difficulty in obtaining, accessing, and/or
compiling the data.
Lack of Knowledge
Lack of understanding about data
analytics.
Lack of Management Buy-in
Management not clear on scope, value, or
outcomes.
Insufficient Resources
Insufficient Resources or the need to train
personnel.
Results Interpretation
Inability to interpret results and
communicate effectively.
Time Commitment
Time required to develop and execute
analytical procedures01
03
05
02
04
06
Source: The 2015 Data Analytics and Leadership Survey (The IIARF and Grant Thornton)

Intelligence for Inspection Readiness
When FDA contacts you regarding an Inspection:
• Do you have a plan to get ready?
• Does your plan include Data Analysis?
• Do you worry FDA is going to find something you didn’t?
• Do you know your vulnerabilities?
• How do you know your ready?
• How do you measure your readiness state?
• What Quality Management Subsystems do you focus on for Inspection
Readiness?
• What data do you review first?

Intelligence Strategies for Inspection Readiness
• Routinely Analyze OPEN FDA DATA
• Be aware of your FDA risk profiles
 Quality Profile – Recalls, MDR’s
 Compliance Profile – 483’s, Warning Letters, Ext. Audit Data
• Benchmark your MDR and Recall reporting:
 By Product Code
 By Competitor
• Prioritize Product Problems and Device Malfunctions for Design Change
• Stay on top of regulatory compliance trends
• Review Internal Data for Improvement Opportunities and Compliance exposure
• If you don't know where to start, start with digitizing Internal Audit data!

Assessing Internal Compliance Risks
P&PC
Production and Process controls
are a major contributor to FDA
compliance Gaps
Training
FDA is focusing more and more
on Training Effectiveness as a
CAPA activity.

CAPA
Its important to understand how
effective the CAPA process
performs over time.
Internal Audit
Outstanding Internal audit gaps
may be an indication of additional
compliance risk.

Document Controls
Monitoring ECO’s can be used as
an indicator of design
robustness.

Assessing External Compliance Risks
Complaint Aging
Complaint Aging may be
pointing to other areas of the
quality system that need
attention.
Complaint Trending
Complaint Trends may
highlight Product
Performance concerns in
the field.

Assessing External Compliance Risks
MDR’s
Late MDR reporting may
lead to undesirable 483’s.
Recalls
Device Recall analysis
can help in understanding
Total Product Quality
Risks.

Device Intelligence Use Cases for Quality Improvement
External FDA Data Sources Internal Data Sources Business Value
MAUDE – Product Malfunctions Design Control - Design Inputs Analysis of Product Problems for similar devices
by Product code for product design
MAUDE – Events Risk Management – DFMEA Evaluation of Probability of Occurrence against
Severity of Event Types and reported
malfunctions
TPLC Design Control - Design Changes Analysis of Design Changes against reported
Device Problems including Recalls
NIST – National Vulnerability Database
(Cyber Security)
Risk Management – Software Hazard
Analysis
Analysis of Exploitability of Software
Vulnerabilities for Cyber Security against Product
Safety
NIST – National Vulnerability Database
(Cyber Security)
Software Device Verification & Validation Comparison of NIST Vulnerabilities against
Internal V&V Penetration & Fuzz Testing results
FDA 483’s Quality Audits Comparison of FDA 483’s against Quality Audit
Analytics for Inspection Readiness Preparation
MAUDE – Product Malfunctions Software Design Verification & Validation Analysis of Software Defects reported (escapes)
against internal V&V Defect Density

Questions?
Contact Info: Armin Torres
e-mail: atorres@qualifiedsystems.com
Web: www.qualifiedsystems.com

FDA News Webinar - Inspection Intelligence

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (17)

Similaire à FDA News Webinar - Inspection Intelligence

Similaire à FDA News Webinar - Inspection Intelligence (20)

Dernier

Dernier (20)

FDA News Webinar - Inspection Intelligence