CRO/Vendor oversight should support sponsor regulatory requirements and cost containment.Quality Management in clinical operations are Centralized Monitoring, Study Quality Metrics and CRO Oversight.

1. Webinar

2. 2
Agenda
1
Recognize the importance of CRO/Vendor
oversight
Recognize
2
Identify ways to integrate a risk-based
approach to oversight
Identify
3
Developing an oversight plan with proper
KPIs/metrics to be measured
Develop
4
Use of tools and technology for accurate and
timely reporting
Review

3. 3
Our Speaker - Sandra SAM Sather
MS, BSN,
CCRA, CCRC
Clinical Operations
& Risk Expert
Over 25 years of
clinical ops &
training experience

4. 4
Sponsor CRO oversight should include risk assessment
that includes risk identification at a quality system and
project level, and risk prioritization based on the
impact, likelihood of error, and the extent to which
error would be detectable for identified data and
processes.
Project Risk Assessment: The Use of a CRO is Part of
the Sponsor Project Risk Assessment
• When is this performed?
• By whom?
• How often?
• Actions Taken?
• Documentation?

5. 5
November 2013, EMA
Final Reflection Paper on Risk Based Quality Management in
Clinical Trials
• Purpose of quality management procedures to:
 Provide assurance that the rights, safety and well-
being of trial subjects are protected
 Ensure the results of the clinical trials are credible
• Same requirements apply to others to whom the sponsor
has delegated any trial related duties and functions of the
sponsor:
 Contract Research Organizations (CROs)
 Vendors or other service providers

6. 6
Risk-based Approaches Related to
Sponsors Working with CROs
Within the FDA Guidance RBM: VI. C. Delegation of
Monitoring Responsibilities to a CRO
• [Teaching moment] regulatory requirement for
transfer (drug, refer to the FDA CPGM related to
device and CROs) and responsibility to retain
OVERSIGHT.
• Prospectively consider additional Factors.
• Risk assessment communications bi-laterally of
study risk [and risk assessment of CRO
performance]
• Risk monitoring and re-assessments clear roles and
responsibilities.

7. 7
Updates to the FDA Final RBM Guidance
related to CRO/Vendor Management Impact
• Impacts vendor management, delegating
monitoring obligations to a CRO
 Requires more clarity, communication,
and evaluation than traditional site
monitoring.
 Monitoring is not just the monitor, but
quality oversight.
• Supports that FDA has communicated goals of
RBM to FDA staff and in the BIMO Compliance
Program Guidance Manuals (CPGMs).
• Like EMA, focus on quality risk management in
sponsor clinical research systems.

8. 8
EMA Final Reflection Paper on Quality
Risk Management (QRM)
Nice Lists of Key Elements of a Clinical Quality System:
 Documented procedures and validated methods being developed,
implemented and kept up-to date
 Documentation system that preserves and allows for the retrieval
of any information/documentation (quality records/essential
documents) to show actions taken, decisions made and results
 Appropriate training of sponsor personnel as well as of the
personnel in affiliates, at the CROs, vendors or other service
providers and at trial sites
 Validation of computerized systems
 Quality control, for example monitoring of trial sites and central
technical facilities on-site and/or by using centralized monitoring
techniques
 Quality assurance including internal and external audits performed
by independent auditors

9. 9
• Sponsor current quality systems include updates to support
QRM and oversight related to vendor
management/oversight.
• What is expected from the CRO/vendor related to their
quality systems and risk assessment.
• Addressing current practices that are not proportionate to
the risk nor well adapted to achieving project goals (refer
to the slide notes and also EMA Reflection Paper page 5)
 Avoiding micro-management vs. oversight
 Base the balance on risk assessment
 What areas need improvement at sponsor level?
 What areas are most impacting if CRO lacks or
weak quality system component?
EMA: QRM Requirements Noted that
Specifically Apply to Vendor Management

10. 10
Common Disproportionate Areas
• Expectation: move towards a more systematic and risk-based approach
 Move to a more proportionate approach adopted to the risk of the
project.
 Target limited resources better.
• Both FDA and EMA are asking sponsors and those that perform sponsor
requirements/ tasks to take QRM practices and clinical quality systems from
GMP to GCP. ICH Q9 (QRM) and Q10 (QS)
• So, do sponsors increase risk to your CROs/vendors by poor design of
protocol, or over interpretation of the regulations?

11. 11
New QRM Requirements’ IMPACT
• Applies to those that perform sponsor GCP functions
• Safety, data management, monitoring, etc.
• Not just the Monitor
• But also affect the selection of CROs/Vendors
• e.g., Request for Information (RFI), Request for Proposal
(RFP), contracts, transfer of responsibilities (TOR), scope
of work (SOW).
• And oversight of CROs /vendors by a sponsor
• e.g., project deliverables, project reports,
communication, etc.
• Adjust clinical processes related clinical QRM
 Vendor assessments and evaluations
 RFI and RFP
 Escalation policy
 Risk assessment of vendor involvement
 Communication plans

12. 12
Flow of QRM in GCP
PLAN
DOCHECK
ACT

13. 13
Quality Risk Management (QRM)
Respond to deviations
Measure/monitor Study conduct
Identify quality objectives and metrics
and risks to quality to develop quality
management plans
(e.g., Vendor Oversight Plan, CRO
Project Plan)

14. 14
Integrating Risk Management Process
Flow with Flow of GCP Vendor Oversight
[ISO/ ICH]
I M P A C T A N A L Y S I S
• Cross-functional
team
• Risk Management
Leader(s)
• Risk Tool /
Platform
• Risk Assessment
(Compliance,
Financial, etc.)
• Link to historical
data from metrics
/ audit findings
Organization Level
QRM
Pre-study
Develop Study Risk
Management Plan
(MP, DMP, SMP, etc.
▪ List of potential
threats
▪ Risk assessment
• Product
• Disease
• Patient
• Self
• Sponsor
• IRB
During the Trial
Plan(s) Management
 Timeliness of data
completion
• Enrollment
• AE
• Deviation
• Audit results
• Noncompliance
• Staff turn-over
• Risk mitigation
adjustment
Study Closure
• Enrollment
outcomes
• Deviations
outcomes
• Audit readiness
• Metric adjustment
from Risk
Management Plan
statistics
Re-Assess & Adjust

15. 15
(EMA) STEP I: Risk Identification
2 levels considered when gathering information for
clinical trial risk identification (EMA, 2013)
• 1st level = system risk
 Information associated with the
environment and its systems should be
analyzed to identify potential risks that
could affect organizations, their technology
and their data and products.
 Indirectly affect a clinical trial.
• 2nd level = project risk
 Information directly linked with the trial
should be analyzed to identify the risks
that are trial specific
PLAN

16. 16
1st level = Systems Risk
EMA, Nov 2013:
 organization structures and responsibilities (e.g. organograms,
communication plans, contractual partners)
 quality systems and processes (e.g. standardized procedures)
 facilities and computerized systems (e.g. Information technology
infrastructure, document management system, data management
system, IVRS, eCRF system)
 human resources including training and qualifications of personnel
(e.g. job descriptions, training plans, performance management)
 compliance metrics, performance measurements, quality audit
and/or inspection outcomes
 regulatory and ethical framework (e.g. knowledge of national and
local approvals and notification required and their timelines)
Includes Sponsor & CRO system risks
• RFI / RFP
• SOW & Project Plan(s)

17. 17
2nd level = Project / Trial Risk
EMA, November 2013: Single clinical trial or full development program
• investigational medicinal product(s) [IMP];
• trial design and protocol specific requirements;
• operational:
 study budget,
 deadlines,
 staff numbers and talent,
 project management (e.g., multi-vendor),
 resources and the training (e.g., outsourcing to CRO),
 equipment and technology,
 trial or product specific procedures/methods
Includes Sponsor & CRO system risks
• RFI / RFP
• SOW & Project Plan(s)

18. 18
For CRO Management: What Are the
Areas of Greatest Risk & Likely to Occur?
Sponsor contributed
• System Level
• Project Level
• Pre-study
• During
• End of
study
CRO contributed
• System Level
• Project Level
• Pre-study
• During
• End of study
Early meeting with the CRO to do a project risk assessment. Risk Assessment
to Fishbone the 6 main areas of risk (that rank high enough based on severity
and likelihood, timing), and identify root cause and verify interventions and
detection mechanisms are in place.

19. 19
• For a study where a sponsor has contracted a CRO to do
study monitoring, who should perform the study/project
risk assessment?
A. Sponsor
B. CRO
C. Both
D. Neither
E. Depends
F. Not Sure

20. 20
• For a study where a sponsor has contracted a CRO to do study
monitoring, who should perform the site risk assessment?
A. Sponsor
B. CRO
C. Both
D. Neither
E. Depends
F. Not Sure

21. 21
Step II: Establishing Priorities for Risk
Evaluation
PLAN
Project Risk Assessment:
• Does the sponsor prioritize / evaluate for the CROs
related to their assigned / contracted sponsor
responsibilities? Or is this micro-management.
• Or does the sponsor review and approve the CROs step
II? How would this be communicated and overseen?
 Ideally this should be the CROs job.
Step II for sponsors oversight systems
• Be sure that prioritization of sponsor CRO oversight
activities are based off actual risk assessment, not
because we “always do it a certain way” or no way.
• Oversight activity prioritization.

22. 22
Step III: Risk Control
DO
• Sponsors evaluate if CROs do this related to their assigned
/ contracted sponsor responsibilities (SOW).
 How would this be communicated and overseen?
• For sponsors oversight systems involve risk control related
to CRO oversight, not performing sponsor responsibilities,
unless from assessment that is the safest alternative.
• Sponsors evaluate if CROs do this related to their assigned
/ contracted sponsor responsibilities.
 How would this be communicated and overseen?
 For sponsors oversight systems involve risk control
related to CRO oversight, not performing sponsor
responsibilities, unless from assessment that is the
safest alternative.

23. 23
Risk Control: Alternative Oversight
Techniques
DO
• Communication with Vendor / CRO
– Encourages various modes of communication at
Specific study time points and activities.
 Work products linked to mitigation interventions
and metrics for tolerance levels.
• Review of CROs Processes, Procedures, and Records Using
Central Monitoring Techniques
– eTMF
– CTMS
• Milestones and Critical Data Points
– Define what are critical study parameters and how will
they be reported vs. all data.
– Clearly define what is pertinent info to support oversight
not micromanagement.
– Does NOT decrease the requirements of sponsor
regulatory application responsibilities

24. 24
Step IV: Quality Tolerance Limits
CHECK
• Who establishes tolerance limits? Data vs.
Oversight
• Who tracks, evaluates root cause of
deficiencies / violations and intervenes?
• Who determines metrics? Good to ask what
are quality metrics used by CRO during RFI
and/or RFP. Traditional vs. post RBM.

25. 25
Quality Tolerance Limits continued…
Examples of areas for which variation or tolerance limits could be
established and linked to the CRO reports to Sponsor: 1) Trial data,
2)Trial protocol procedures and GCP 3)Trial management procedures
 Define the metrics that will allow oversight of the trial
 Establish the oversight strategy
 Include escalation plans and communication strategy to ensure
quality tolerance triggers lead to sponsor notification
(reading MVR, outlying performance, change in staff, etc.)
 Sponsors should have established action plans for quality
tolerance deviations (e.g., may be part of a vendor oversight
plan, or within the quality risk management plan)
 Define the timing of CRO sponsor reporting/retrieval of data

26. 26
Step V: Risk Review Cycle
ACT
Concept of risk based clinical quality management revolves
around the following cycle
1. risk assessment with information gathering, the
establishment of priorities and the identification of risks
associated with the study;
2. risk control which encompasses setting tolerance limits
mitigation and acceptance of risks;
3. risk review which necessitates knowledge of the previous
steps with the integration of the risk management tools
and the communication on the review of the results and
data associated to the risk identified and the
documentation of the actions needed.

27. 27
Summary
• Recognize the importance of CRO/Vendor oversight in clinical trial
management.
• Identify the ways to integrate a risk-based approaches to CRO/Vendor
oversight.
• Developing a CRO/Vendor oversight plan with appropriate KPIs/metrics to
be measured.
• Use of tools and technology for accurate and timely reporting.
• Steps of CRO/Vendor Oversight with a Risk-based Approach
 Step I Risk Identification
 Step II Risk Prioritization
 Step III Risk Control
 Step IV Quality Tolerance Limits
 Step V Risk Review
 Step V Reporting Quality

28. Metrics …

29. 29
Balancing Dilemma ..
What gets
measured gets
fixed
Analytical mindset
When there is too much
information to process, we
tend to use “common
sense,” or “educated
guesses.” But, when we use
common sense, we tend to
be biased.
Logical mindset
Metrics that help organizations overcome common challenges like excessive
workloads, rework, unclear or changing requirements, and disappointing
outcomes in services, products and finances.

30. 30
Right mix
of Types
Measure Processes,
not People
Big picture in mind
Define.
Define. Define
Start Simple
Through out the process, keep asking “so what”
Purpose of Measurement ?
Answer this first before starting any steps …
TAT, Quality
Cycle Time, Efficiency
Leading & Lagging Indicators
Dive into calculations. Be on same page.

31. B
A
C
Output
from C -> D
Output
from A->B
Output
from B -> C
D
Multiple Vendors… your View

32. ..their View

33. Operational
Level
Agreement
OLAs are agreements
between different
Vendors (and/or your
internal business
functions)
(where as, Metrics, also
known as SLAs, are the
overall agreement
between Vendors and
your Organization)
A
B
C
D
...time bound Accountability

34. 34
Quick Comparison
SLAs address the Business Level Requirements, e.g. "After
Agreements signed, FPI completes in 8 weeks".
For business it is not of interest, how many days are needed by
the different vendors, e.g. CRO vendor for site negotiations, EDC
vendor for DB live, Patient recruitment vendor to start the
advertisements.
These details must be defined in the OLAs which the
Organization (can be represented by an Outsourcing manager or
an Alliance manager or Procurement or Program/Project
manager) needs to negotiate and agree with all involved
vendors.

35. 35
Data-driven Decision making
You do not have time to spend extracting information
from several reports from several vendors (and
sometimes poorly designed or inappropriate Charts).
Performance metric reports should highlight critical
data and provide the user (that’s you) with the
confidence to make decisions that will improve
processes and deliver results.
Key is to have this info come
to you INTEGRATED..
.. and in REAL TIME

36. 36

37. 37

38. 38

39. 39

40. 40
Joint CommitteeYou Vendor
Clearly structured model to steer the partnership
for a successful collaboration between YOU and VENDOR
Strategic
(quarterly)
Alliance
Committee
Your
Exec
Management
Vendor
Exec
Management
Tactical
(Daily business)
Support &
Ops Leads
Development
Ops Teams
Project
(monthly)
Operations
Committee
Program
Management
Project Manager
&
Functional Leads
Reporting
Governance

41. Thank You
for your Attention !!
Additional Questions, please
email to info@ddismart.com