2014_NVF_III_Session_3_Risk_Assessment.pdf

Session 3
Validation, Risk and
Criticality
Presented by Trevor Schoerie
14 May, 2014

© PharmOut 2014
Validation, Risk and Criticality
• In process validation we want to
understand and control the variation
not the risk.
• The “drug” benefit vs risk should
have already been determined,
(hopefully).
“GAMP 5 – Risk Based ……”
“We talk about QRM every time we meet!”

© PharmOut 2014
“Process Variation”
1. Sample Variation
2. Analytical Variation?
3. Process Variation – normally associate
Is the sample representation of the batch?
Blinded samples in lab = STD DEV
Real process variation

© PharmOut 2014
This session
Using Risk to determine no of batches
Risk vs criticality

© PharmOut 2014
How we get to validation…..
• New Chemical Entity *Stage 1 data
• Generic manufacturer
• Contract manufacturer
• New manufacturing site – Tech. Transfer
• Country where validation is a new
requirement
• Expansion of regulations, i.e. export to
Australia
No concurrent release of validation batches
No retrospective validation

© PharmOut 2014
Typical validation NCE – Stage 2a/b?
Real life
Start
Registration /
QbD

© PharmOut 2014
How many PPQ batches?
QTPP
CQAs/
CPPs
Process
Description
Analytical
Methods
SOPs &
Batch
Records
Design
Reports
FSE
Qualified
PVMP
Training
Completed
Approved
PV
Protocols

© PharmOut 2014
Knowledge vs # of PV batches
Prior Knowledge Process Design
Prior
Knowledge
PV
Process Design PV
Comprehensive Prior Knowledge may support fewer PV batches
Limited Prior Knowledge may require more PV batches

© PharmOut 2014
This depends on the risk and the following elements:
Based on science, experience and
justified (documented).
Based on statistics.

© PharmOut 2014
Step 2:
Risk Assessment of Control
Strategy
Step 3:
Determine Overall Residual
Risk
Step 4:
Translate Overall Residual
Risk into # of PPQ Batches
Stage 1
Assessment
Stage 2
Activities
Prepare PPQ Batches
Compare results to
Acceptance Criteria
Risk Level Acceptable
Step 1:
Risk Assessment of Product
Knowledge & Process
Understanding
Reference ISPE

© PharmOut 2014
Risk Assessment of Stage 1
Product Knowledge
• Analytical understanding
of product structure
• Mechanistic
understanding of product
profile
• Contextual understanding
of CQA’s
Process Understanding
• Depth of understanding
unit operations
• Level of knowledge of
process response to input
variability
• Process predictability
• Understanding of scale
Control Strategy
• Application of
process/product
understanding to
production controls
• Control of raw material
inputs
• Equipment capability vs
process requirements
Assess each CQA
Can use:
• Risk Charts
• Relative level of
risk based on
RPN
• Other QRM tools

© PharmOut 2014
Step 1: Risk Assessment of Product
Knowledge
Identification of CQA and impact of CQA
variation on patient:
• Level of understanding of product attributes
(i.e. how a particular attribute affects patient
safety & efficacy)
Product Characterisation:
• The strength of the link between the CQA
and clinical performance

© PharmOut 2014
Product Knowledge Risk Ranking
Product
Knowledge
Factor
Relative Risk Ranking
Characteristics of ranking assignments
Low Risk Medium Risk High Risk
Identification of
CQA and impact
of CQA variation
on patient
• Physiochemical/
biological,
pharmacokinetic
knowledge, and QbD
approach used to
design the formulation
of drug product
• Impact of variation on
bioavailability explored
& understood
• CQAs identified and
justified
• Physiochemical/
biological,
pharmacokinetic
properties identified
• Some exploration of
impact of variation
• Product
specifications
established from
development trial
and error
• Impact of variation
known only from
evaluation of
incidents
Product
Characterisation
• Analytical method has
direct measurable
linkage to clinical
performance
• Complete product
• Analytical method
development based
on mechanism of
action for the
therapeutic agent but
linkage to clinical
performance
• Product
characterisation
measures quality
against established
empirical limits

© PharmOut 2014
Process Understanding Risk Ranking
Process
Understanding
Factor
Degree of
process
understanding/
unit operation
-First principles
understanding: based on
an understanding of
prevailing mechanisms and
rationale
-Casual knowledge: that
based on what causes
interrelationships
between variables
-Descriptive
knowledge: derived
only from observation,
reflecting basic facts
Process
predictably and
modelling
-Models based on first
principles
-Extension of empirical and
mechanistic models
-Highly predictable process
and scale-up
-Use of models derived
from basic physical,
chemical, biological or
microbial mechanisms of
observed phenomena
-Sufficient knowledge to
employ PAT methods, if
applicable and desired
-Primitive models
reflecting only basic
understanding of
process and scale
effects
-Process predictability
is questionable
Table continued on the next slide

© PharmOut 2014
Process Understanding Risk Ranking
Process
Understanding
Factor
Process Response
to input variability
-Design space identified
using multivariate data and
statistical methods
-Impact of material
attributes on product
quality explored extensively
in development
-Material specific CQAs
identified and well
understood or no material
specific CQAs
-Well-defined criticality
for process based on
multivariative
experiments
-Impact of material
attributes on product
quality explored to some
degree
-Material specific CQAs
identified-full range of
variability not explored in
development
-Partially defined,
primarily through
univariate
experimentation
-Impact of materials
attributes to product
quality are minimally
explored
-Materials specific CQAs
not identified
Effects of scale -Highly predictable-data
across different scales can
be projected.
-Predictable-data across
scales can be projected
-Unpredictable-Scale
impact

© PharmOut 2014
Step 2:
Strategy
Step 3:
Risk
Step 4:
Stage 2 Activities
Prepare PPQ Batches
Compare results to
Acceptance Criteria
Step 1:
Knowledge & Process
Understanding
Stage 1
Activities
Assessments of
Product Knowledge
and Process
Understanding

© PharmOut 2014
Step 2: Risk Assessment of Control
Strategy
• How the specifications were developed.
Raw Materials
Specifications
• How easily the requirements for the process are
accommodated by the manufacturing equipment.
Equipment Capability
vs. Process
Requirements
• How consistently the process has performed
historically and during development studies.
Experience with
Process Performance
• How the process is monitored and variability is
detected
Monitoring capability
& detectability

© PharmOut 2014
Control Strategy - Risk Ranking
Control
Strategy
Factor
Source of
Potential
Variability and/or
Uncertainty
Raw Material
Specifications
-Different suppliers;
different
manufacturing
processes,
-Material attributes
test method
-Different batches
-Basis for material
specification
-Specification wider
than experience
-Specifications of
material attributes
impacting product
quality based on
development data
-Limited
justification of
specifications of
material attributes
-Specifications are
not justified.
-Compendial or
supplier limits
accepted without
further
investigation

© PharmOut 2014
Control
Strategy
Factor
Source of
Potential
Variability and/or
Uncertainty
Equipment
Capability vs.
-Capability of
equipment to
-Comparison of
parameter control
-Comparison of
control ranges
from
-Comparison of
parameter control
Process
Requirements
-Control operating
parameters within
acceptable ranges
-Ranges from
equipment
qualification with
process
requirements
indicates all
parameters are
well within
equipment control
capabilities and
supported by
qualification data
-Equipment
qualification with
process
requirements
indicates marginal
capability to meet
requirements for a
limited number of
process
parameters
-Ranges from
equipment
qualification with
process
requirements
indicates a
significant number
of parameters are
similar to
equipment control
capabilities

© PharmOut 2014
Control
Strategy
Factor
Source of
Potential
Variability
and/or
Uncertainty
Experience
with process
performance
to date
-Variation
observed
-Scaling
effects
Consistency
of past
performance
-Underlying cause(s) for
variation is understood
& addressed (or
variation not observed
during manufacture)
-Impact of scale is well
understood
-Process has
consistently performed
as expected
-Variation is managed
empirically, but
underlying causes are
not well understood
-Some understanding
of scaling issues
-Minor departures
from expected results
that were investigated
and satisfactorily
explained
-Variation has
been observed,
but has not been
successfully
managed
-Impact of scale
changes has not
been explored
-Unexplained
failure has been
experienced
Monitoring
capability &
detectability
-Ability of
monitoring
tools &
methods to
detect
variation
-Attributes measured in
real time at a sensitivity
where performance
variability is likely to be
observed
-Attributes measured
off-line (after batch
completion) at a
sensitivity where
performance is likely
to be observed
-Attribute
measurement
accuracy is
inadequate

© PharmOut 2014
Stage 2
Step 2:
Strategy
Step 3:
Risk
Step 4:
Stage 1
Activities
Assessments of
Product Knowledge
and Process
Understanding
Stage 2 Activities
Prepare PPQ Batches
Compare results to
Acceptance Criteria
Step 1:
Knowledge & Process
Understanding

© PharmOut 2014
Step 3: Determine overall residual risk
The residual risk level reflects the confidence in
performance of the commercial process:
Residual Risk
Level
Description
Severe
(5)
Multiple factors have high risk ratings
High
(4)
Few factors have high risk ratings or all have
medium risk rating
Moderate
(3)
Medium risk level for multiple factors or high risk
level for one factor
Low
(2)
Medium risk level for a few factors, the others are
low risk
Minimal
(1)
Low risk level for all factors

© PharmOut 2014
Step 2:
Strategy
Step 3:
Risk
Step 4:
Stage 1
Assess
Stage 2 Activities
Prepare PPQ Batches
Compare results to
Acceptance Criteria
Step 1:
Knowledge & Process
Understanding

© PharmOut 2014
Step 4: Translate overall residual risk
into the number of PV Batches
1. Based on rationales and experience
2. Target Process Confidence and Target Process
Capability
• Statistical-includes measures of variability &
confidence level
3. Expected Coverage
• Statistical-includes measure of probability of
batch success rate
Note: Other approaches may be appropriate

© PharmOut 2014
How many PV batches?
Approach 1:
Rationales and
experience
Approach 2:
Target process
confidence &
target process
capability
Step 4:
Translate
Overall
Residual Risk
into # of
PPQ Batches
2 Approaches to
Translate Overall
Residual Risk into
# of PV Batches

© PharmOut 2014
Approach 1: Rationales and Experience
Residual
Risk Level
Number of
Batches
Rationale
Severe
(5)
Not Ready
for PV
Encourage additional development to reduce risk level
High
(4)
10 High # of consecutive successful batches unlikely if
controls are not adequate
Moderate
(3)
5 Increased residual risk addressed by preparing 2
additional PV batches to provide further
demonstration of process consistency
Low
(2)
3 Knowledge & Control Strategy regarded as sufficient.
3 PPQ batches has historically been appropriate for
demonstrating process consistency for many low-risk
processes
Minimal
(1)
1-2 Minimal residual risk with less than 3 PPQ batches
required, e.g. for verifying specific controls associated
with a well-understood change to a process

© PharmOut 2014
Approach 2: Target process confidence
& target process capability
• Process Capability (CpK) is used as a measure of
the capability of the process to consistently meet
the quality requirements
• Assumption: CpK ≥1 as a starting point for
assessing the capability of a process undergoing
validation

© PharmOut 2014
• Used as a measure for level of confidence
needed in the CpK and thereby as a degree of
assurance
• High level of confidence in the CpK can be built
only with time and experience (during Stage 3)
• Residual Risk Level used to define the
confidence needed at completion of Stage 2.

© PharmOut 2014
Residual
Risk Level
Target
Confidence
Comments
Severe (5) N/A Major gaps in knowledge & understanding.
Additional effort on product/process/control strategy
development may be necessary. High confidence
level needed to provide high degree of assurance.
High (4) 97%
Moderate
(3)
95% Target confidence levels designed to provide
reasonable assurance of process capability. Higher
confidence levels would be achieved during Stage 3.
Low (2) 90%
Minimal (1) N/A High “confidence” based on existing understanding
and capability of control strategy. Does not require
additional assurance during PV beyond
demonstration that commercial systems and
procedures are appropriate.

© PharmOut 2014
Residual Risk
Level
Min # of
batches
Target
Confidence
for CpK 1.0
Acceptance Criteria
Readily Pass
Calculated
CpK
Marginally Pass
Calculated CpK
Fail
Calculated CpK
Severe (5) Not Ready
for PPQ
N/A
High (4) 11 97%
≥ 1.6
≥ 1.0
and
< 1.6
< 1.0
Moderate
(3)
8 95%
Low (2) 5 90%
Minimal (1) 1-3 N/A
• Based on a Target CpK of 1.0 and an actual CpK of 1.6.
• May include clinical manufacture, demonstration, or other at-scale lots.
Assumes process will be under statistical control & data will show
normal distribution.

© PharmOut 2014
PPQ Outcome Pass / Fail Other
Considerations
Impact on Initial CPV
Sampling Approach*
Readily Pass
calculated CpK
Pass N/A Supports Stage 3 routine
sampling
Marginally Pass
calculated CpK
Pass All input/output
parameters within
range
Consider enhanced
monitoring for CQA’s not
meeting “Readily Pass”
Fail calculated
CpK
Investigate -Parameter values
-Intra-lot CpK
-Probability of
detection
-Process
improvement
options
Consider enhanced
monitoring; May include
some testing beyond
PPQ.
*PPQ outcome is one consideration in establishing CPV Plan

© PharmOut 2014
FMEA - recap
• Study the Failure Mode and Effect Analysis
• It involves reviewing as many
components, assemblies, and
subsystems as possible to identify
failure modes, and their causes and
effects.

© PharmOut 2014
Ishikawa – Cause and Effect Diagram

© PharmOut 2014
Quality Risk Management (QRM)
• This session will take a step back and look at the
basics
• We will ponder some topics:
• Is the “Criticality Assessment” a “Variability
Assessment” when discussing CQAs/CPPs for
Qualification & Validation activities?
• Do we throw out our innate understanding of the
hazard because we are “FMEA” driven?
FMEA does not help identify sources of
variation

© PharmOut 2014
ICH Q9 Quality Risk Management (&
Annex 20 of PIC/S PE 009-8)
• ICH Q9 explains the “What” of QRM [10 pages]
• Annex I of ICH Q9 provides concepts and ideas on the
“How to”-formal and informal [4 pages]
• Annex II of ICH Q9 details the potential “Where” of
QRM [5 pages]
• A “roadmap” is presented to us within the document,
but do we follow it for Qualification & Validation
activities?
• For example, do we make use of QRM tools other than
FMEA (or FMECA) effectively?

© PharmOut 2014
• Risk is defined as the
combination of the
probability of occurrence
of harm and the severity
of that harm
• Harm: Damage to health,
including the damage that
can occur from loss of
product quality or
availability.
• Severity: A measure of
the possible consequences
of a hazard.
Begin the QRM Process
Risk Identification
Risk Analysis
Risk Evaluation
Risk Reduction
Risk Acceptance
Risk Assessment
Risk Control
Output of the QRM Process
Review Events
Risk Review
QRM
Tools
Risk
Communication

© PharmOut 2014
Wet Floor
(Hazard: The potential
source of harm)
Broken Leg
(Harm: Damage to health, including
the damage that can occur from loss
of product quality or availability)
Severity: A measure of the possible
consequences of a hazard.
Risk is defined as the combination of the probability of occurrence of harm
and the severity of that harm

© PharmOut 2014
Deductive vs Inductive Reasoning
Destroyed my car.
How did this happen?
Intoxicated?
What will happen if I
drive?
Inductive (forward logic)
Deductive
?
FMEA
PHA
FTA
STA
C&E Analysis

© PharmOut 2014
Inductive Reasoning
Outcome
Hypothesis
Observation
Confirmation
Risk
Identification
Risk Analysis
Risk
Evaluation
Risk
Reduction
Risk
Acceptance
“Top-down"
“Think up”
Failures!
More Proactive?

© PharmOut 2014
Deductive Reasoning
Theory
Tentative
Hypothesis
Pattern
Observation
Risk
Identification
Risk Analysis
Risk
Evaluation
Risk
Reduction
Risk
Acceptance
“Bottom-up"
“Narrow down”
Failure!
More Reactive?

© PharmOut 2014
Inductive vs Deductive QRM
QRM Tool Inductive Deductive “Complexity”
FMEA Yes No Medium
FMECA Yes No Medium
FTA No Yes High
HACCP Yes Yes Low
HAZOP Yes No Medium
PHA Yes No Low
Risk Ranking and
Filtering
Yes Yes Low
Supporting Statistical
Tools
Yes Yes Low

© PharmOut 2014
Appendix I: Risk Management Methods
& Tools
• General overview-references some primary tools
“It is important to note that no one tool or set of tools is
applicable to every situation in which a QRM procedure
is used”
FMEA FMECA FTA HACCP
HAZOP PHA
Risk Ranking
& Filtering
Supporting
Stats Tools

© PharmOut 2014
& Tools
QRM Tools Input Process Output
FMEA Relies on product
& process
understanding.
Manageable
process steps.
Once potential failure
modes are established, risk
reduction can be used to
eliminate, contain, reduce
or control the potential
failures.
Summarises modes of
failure, factors causing
these failures and the
likely effects of these
failures.
FMECA Relies on product
& process
understanding.
Manageable
process steps.
Once potential failure
modes are established, risk
reduction can be used to
eliminate, contain, reduce
or control the potential
failures.
The output of an FMECA
is a relative risk “score”
for each failure mode,
which is used to rank the
modes on a relative risk
basis.
FTA System (or sub-
system) failures
Evaluates system (or sub-
system) failures one at a
time but can combine
multiple causes of failure
by identifying causal
chains.
The output of an FTA
includes a visual
representation of failure
modes.

© PharmOut 2014
Failure Mode Effects Analysis (FMEA)
Item or
process
Step
Potential
Failure
Mode
Potential
Effect(s)
of Failure
Severity
Potential
Cause(s)
Occurrence
Current
Controls
Detection
RPN
Recommended
Action
Responsibility
& Target Date
Action
Taken
Severity
Occurrence
Detection
RPN
FMECA: Extended to incorporate an investigation of the
degree of severity of the consequences, their respective
probabilities of occurrence and their detectability

© PharmOut 2014
& Tools
HACCP Product and
process
understanding.
It is a structured approach
that applies technical &
scientific principles to
analyze, evaluate, prevent,
and control the risk or
adverse consequence(s) of
hazard(s)
Risk management
information that
facilitates monitoring of
critical points not only in
the manufacturing
process but also in other
life cycle phases.
HAZOP All processes &
safety hazards.
It is a systematic
brainstorming technique
for identifying hazards
using so-called “guide-
words”.
As is the case with
HACCP, the output of a
HAZOP analysis is a list of
critical operations for risk
management.
PHA Product, process
and facility design
information
1) Identification, 2)
Evaluation 3) Ranking, and
4) Remediation
Typically, hazards
identified in the PHA are
further assessed with
other risk management
tools such as those in this
section.

© PharmOut 2014
Preliminary Hazard Analysis (PHA)
• Early in the development: little information on design
details or operating procedures will often be a
precursor to further studies
• For product, process and facility design
• Further assessed with other risk management tools
Hazards Arising from Product Design
Hazard Investigation/
Controls
Severity
(S)
Frequency
(F)
Impact
(SxF)

© PharmOut 2014
& Tools
Risk
Ranking
and
Filtering
Risk ranking and
filtering can be
used to prioritize
manufacturing
sites for
inspection/audit by
regulators or
industry.
Forms a single relative risk
score that can then be
used for ranking risks.
“Filters,” in the form of
weighting factors or cut-
offs for risk scores, can be
used to scale or fit the risk
ranking to management or
policy objectives.
Risk ranking methods are
particularly helpful in
situations in which the
portfolio of risks and the
underlying consequences
to be managed are
diverse and difficult to
compare using a single
tool.
Supporting
Statistical
Tools
Statistical data They can enable effective
data assessment, aid in
determining the
significance of the data
set(s), and facilitate more
reliable decision making.
Control Charts, Design of
Experiments (DOE),
Histograms, Pareto
Charts, Process Capability
Analysis

© PharmOut 2014
Appendix II: Potential Applications for
QRM
In II.1: Integrated Quality Management
• To interpret monitoring data (e.g., to support an
assessment of the appropriateness of revalidation or
changes in sampling).
• To determine appropriate actions preceding the
implementation of a change, e.g., additional testing,
(re)qualification, (re)validation or communication
with regulators.

© PharmOut 2014
QRM
In II.4: Facilities, Systems & Equipment
• To determine the scope and extent of qualification of
facilities, buildings, and production equipment and/or
laboratory instruments (including proper calibration
methods).
• To determine acceptable (specified) cleaning
validation limits.

© PharmOut 2014
QRM
In II.4: Facilities, Systems & Equipment
To select the design of computer hardware and
software (e.g., modular, structured, fault tolerance);
To determine the extent of validation, e.g.,
• identification of critical performance parameters;
selection of the requirements and design;
• code review;
• the extent of testing and test methods;
• reliability of electronic records and signatures.

© PharmOut 2014
QRM
II.6 Production
• To identify the scope and extent of verification,
qualification and validation activities (e.g., analytical
methods, processes, equipment and cleaning
methods;
• To determine the extent for follow-up activities (e.g.,
sampling, monitoring and re-validation);
• To distinguish between critical and non-critical
process steps to facilitate design of a validation
study.

© PharmOut 2014
Uses in Qualification & Validation
QRM Tools
Tools
Uses in Q&V
FMEA All Q&V Stages-prioritize potential risks and monitor the effectiveness
of risk control activities.
FMECA All Stages-Prioritize potential risks and monitor the effectiveness of
risk control activities. Understand the impact of detectability
FTA Establish the pathway to the root cause of the failure during Q&V. Is
useful both for risk assessment and in developing monitoring
programs as an output of Q&V.
HACCP Used to identify and manage risks associated with physical, chemical
and biological hazards (including microbiological contamination).
HAZOP This facilitates regular monitoring of critical points in the
manufacturing process. Used early in the design stage. Q&V input.
PHA Used early in the development of a project when there is little
information on design details or operating procedures; thus, it will
often be a precursor to further studies. Useful “first cut” for Q&V.
Risk Ranking
and Filtering
Risk ranking is useful when management needs to evaluate both
quantitatively-assessed and qualitatively-assessed risks within the
same organizational framework.
Stats Tools They can enable effective data assessment throughout Q&V activities

© PharmOut 2014
QRM vs Impact Assessment Example
“Top Down”
Critical Quality Attributes
Critical Process
Parameters
System
Subsystem
Component
End
End
Product carryover exceeds
acceptance criteria
Cleaning Agent
Concentration
CIP System
Chemical Addition
Chemical Feed Pump
“Bottom Up”
From ISPE GPG Applied Risk Management for C&Q

© PharmOut 2014
QRM vs Impact Assessment
Characteristics Impact Assessment
(Baseline® Guide 5)
Formal Risk
Assessment
“Top Down” or “Bottom Up” “Bottom Up” “Top Down”
Ability to identify specific process risks Low High
Ability to identify specific Critical
Aspects
Low High
Ability to prioritise Qualification efforts Medium High
Cost in time and resources Medium High
Need for SMEs experienced in the tool
to be used
Low High
Acceptability to regulators Medium High
Usefulness as “lifecycle” document in
future
Low High
From ISPE GPG Applied Risk Management for C&Q

© PharmOut 2014
“Criticality” in Annex 15
• “It is a requirement of GMP that manufacturers
identify what validation work is needed to prove
control of the critical aspects of their particular
operations.”
• “The protocol should specify critical steps and
acceptance criteria.”
• “Evidence should be available to support and verify
the operating parameters and limits for the critical
variables of the operating equipment.”
• “Risk analysis: Method to assess and characterise
the critical parameters in the functionality of an
equipment or process.”

© PharmOut 2014
Going from QTPP to CQA
• We assume that during product
development the clinicians and
toxicologists have established that
these parameters are high risk to
the patient.
• So we are looking at reducing the
variability of these attributes.

© PharmOut 2014
Connecting a CQA to a CPP
• The manufacturer during product
development and based on prior
product and process knowledge
establishes that these process
parameters have a high impact on
variability.
• “Validation” looks at ways to provide
confidence that the variability is
controlled.

© PharmOut 2014
Activity 3: QRM
Please complete the Blue Sheets as a
group and return them to us
Please assign a
Speaker/Scribe at
each table
If there are any
questions,
please ask!

© PharmOut 2014
Thank you for your time.
Questions?
Trevor Schoerie
trevor.schoerie@pharmout.net
Lead Consultant
www.pharmout.net

2014_NVF_III_Session_3_Risk_Assessment.pdf

Recommended

Recommended

More Related Content

Similar to 2014_NVF_III_Session_3_Risk_Assessment.pdf

Similar to 2014_NVF_III_Session_3_Risk_Assessment.pdf (20)

Recently uploaded

Recently uploaded (20)

2014_NVF_III_Session_3_Risk_Assessment.pdf