Mantenimiento

1. Reliability Engineering and System Safety 38 (1992) 67-70
Knowledge engineering approach to risk
management and decision-makingproblems
M. Kitamura
Department of Nuclear Engineering, Tohoku University, Aoba, Sendai, 980 Japan
Application of knowledge engineering techniques is regarded as one of the
promising approaches to resolution of inter-expert conflicts on risk-related
decision problems. Among various causes of the conflicts, differences in
unstated assumptions and in subjective weighting of risk-relevant factors are
treated as the major issues. To identify the causes and to evaluate their
influences, a modified version of analytic hierarchy process (AHP) is proposed
in this paper. The effectiveness of the proposed methodology in the conflict
resolution is demonstrated through several examples.
INTRODUCTION
The method of probabilistic safety assessment (PSA)
is, in principle, applicable to a wide variety of
decision-making problems. In practice, however, the
method has been applied to a restricted class of
problems. One major reason for this restricted
application is that the method often provides us with
results with a large uncertainty. Moreover, the results
conducted by multiple experts often result in an
unrealistically large discrepancy. The credibility of the
PSA methodology is underestimated because of these
empirical observations.
The author believes that the uncertainty and
discrepancy are attributable to technical rather than
essential insufficiencies in the PSA procedure. It
seems possible, at least to some extent, to significantly
alleviate the difficulties by modifying the conventional
procedure. The basic idea is to increase transparency
of the analysis procedure.
A recent benchmark test conducted by JAERI
clearly indicated that the dominating cause of
inter-expert discrepancy is the difference in boundary
conditions, or unstated assumptions, l The benchmark
test also indicated that the discrepancy is significant
even after major assumptions are specified prior to the
analysis because of the effect of other unspecified
assumptions and of selected data. Note that the
selection of data certainly reflects the expert's
assumption on the validity of the data. In this regard,
the difference in the unstated assumptions is identified
as one of the major causes of the discrepancy.
ReliabilityEngineeringandSystem Safety 0951-8320/92/$05.00
(~) 1992Elsevier Science Publishers Ltd, England.
The other plausible cause is the difference in weight
assignment and quality evaluation of risk-relevant
factors. A typical example of such a factor is the
performance shaping factor (PSF) in human reliability
analysis (HRA). For a given task sequence, influence
of various plant-specific PSFs are evaluated by HRA
experts to eventually obtain an estimate of task failure
probability. Selection of an influential set of PSFs and
quality evaluation of the selected PSFs are the key
process in established HRA techniques such as
THERP and SLIM-MAUD. Both the selection (i.e.
weight assignment) and quality evaluation are
conducted on the basis of subjective preference and
thus can be the causes of the inter-expert discrepancy.
It is not practical to try to remove all causes of the
discrepancy. It is quite natural that each expert with
individual educational/occupational backgrounds can
possess different opinions. However, in order to reach
the final solution of the decision-making problem, we
certainly need a method for identifying the essential
causes of the discrepancy. In other words, we need a
method for mutual comparison and evaluation of the
different solution procedures to attain conflict
resolution.
Definition of problem
67
From the above discussion, it is appropriate to
summarize that development of techniques to clarify
the differences in unstated assumptions and in weight
assignment and evaluation of risk-related factors is
desirable to reduce the uncertainty and discrepancy in
the outputs of PSAs. The credibility of predictions
derived by PSA can be enhanced considerably after
such a methodological development.

2. 68 M. Kitamura
The author experienced similar difficulty, inter-
expert discrepancy, during development of expert
systems in various technical fields. The discrepancy is
highly disturbing in conducting knowledge acquisition,
i.e. the most crucial and laborious phase of expert
system development. The main cause of the
discrepancy in this case is also attributable to
differences in unstated assumptions. The technique
successfully introduced to solve the difficulty in
knowledge acquisition is a modified version of analytic
hierarchy process (AHP) methodology. Since the
nature of the difficulty is essentially the same as that
of the PSA area, it seems appropriate to apply the
AHP to alleviate the difficulty in the PSA mentioned
above.
METHOD
Fundamentals of AHP
The AHP is a methodology originally introduced for
assisting decision-making under conflicting require-
ments with multiple criteria.2The wide applicability of
the AHP has been confirmed in many areas.3
Applications of the AHP to PSA-based decision-
making problems can be found in our earlier reports. 4
Only the essential part of the method is explained in
the subsequent paragraphs, since detailed descriptions
can be found in the documents mentioned above.
The basic concept of AHP can be understood in
terms of the sample problem illustrated in Fig. 1. The
problem is to decide the best option for reduction of
human error probability during a transient manage-
ment operation. Assume we have three options:
(A) introduction of a new computerized display
system (CDS);
(B) improvement of inter-crew communication
(ICC) for higher recovery; and
(C) modification of off-site training (OST)
program.
In a conventional HRA/PSA practice, the effect of
these options are evaluated through sensitivity
analysis or case studies. Then, a higher level decision
is made on the basis of the modified HEP value plus
other multiple criteria such as installation cost,
operator acceptance, etc. The AHP decision is carried
out in a more straightforward fashion.
1 Selection of utility indices
The issues to be considered in the decision-making,
i.e. utility indices (UIs) are searched out and listed.
Assume that three UIs, effectiveness (EFFE) in HEP
reduction, implementation cost (COST), and operator
acceptance (ACCE) are selected as the major issues
to be considered. As far as this phase of UI selection
is concerned, the experts are supposed to reach an
agreement.
2 Evaluation of relative weights
The relative weights of these UIs are estimated by the
individual experts. The relative weights are given in
terms of pairwise comparison of UIs. The difference
in expert opinion can be reflected in the results of the
paired comparisons, which are summarized in the
form of a matrix. Denote this matrix as A. For the
(i,j) element of the A, we assign an integer k
(1 < k < 10) if UI-i is judged to be more important
than UI-j. A larger value of k corresponds to a higher
weight of UI-i over UI-j. If UI-i is less important than
UI-j, we assign 1/k to the (i, j) element of A. Taking
the redundancy of paired comparison into account,
only the upper triangular elements of the matrix A are
directly evaluated. The elements below diagonal are
Dec i s i on Hi e r a r chy of AHP
Se e
t
IEffect v
for Redo
(EFTE I
Introduc
New Comp
Display
(CDS)
ction of the
o Reduce Probabi I i ty
Ieness] Cost
ct i on Requ i red
(COST)
tion of
uterized
System
Best Countermeasure
of Scram
II Acceptance
by Operator
(ACCE)
I
I Improvement o f I
Inter-Crew
Communication
(Jcc)
IModification of
Of f--si te Training
Program
(OST)
Fig. 1. Hierarchical decomposition of decision-making problem.

3. Knowledge engineeringapproach 69
determined by the constraint relation; A(i,j)=
1/A(j, i). For N-UIs, the number of paired com-
parison is N(N- 1)/2, and thus becomes three in this
example.
(3) Evaluationof options
The options [CDS, ICC, OST] are evaluated in terms
of each UI. This evaluation is carried out through the
paired comparison of the options, resulting to three
(3, 3) matrices B1, B2, and B3. The preference of
individual experts is reflected at this stage also.
4 Bottom-up aggregationof optionpreferencescores
The results of UI-wise evaluation of options are then
combined together with the UI-weight matrix A
established at step (2) to provide overall evaluation of
each option. The aggregation procedure is straightfor-
ward and can be conducted without any difficulty.
The above procedure can be refined further in
practical applications. The UIs, for instance, are
decomposed into sub-UIs whenever needed. Even
after such modifications, however, the main idea of
the AHP procedure remains unchanged. Through the
top-down decomposition and bottom-up aggregation,
the complex decision-making problem is solved in a
systematic manner.
The advantages of the AHP method over the
conventional techniques of multi-utility decision-
making are summarized below:
(1) The AHP is applicable to non-quantitative,
subjective UI.
(2) The unrecognized decision logic of the expert
manifests itself in the structure of the
comparison matrices.
(3) The AHP allows us to measure the consistency
in weight assignment and option evaluation of
each expert. The measure called consistency
index (CI) is easily obtained as
CI = (~.- N)/(N - 1),
where Z stands for the largest eigenvalue of the
matrix of interest (A, B1, B2 or B3). The
smaller 3. indicates that the paired comparison
is logically consistent.
Modification for expert opinion evaluation
The above statements clearly indicate applicability of
the AHP as a tool to alleviate the technical difficulties
in the PSA-related decision-making problems. The
causes of discrepancy are easily identified by
examining the paired comparison matrices. For each
matrix element with significant inter-expert dis-
crepancy, underlining assumptions should be articu-
lated by the experts. Once the difference in the
assumption is recognized, effort should be made to
reach a common standpoint leading to an identical
assumption. It is usually possible to reduce the
discrepancy through this procedure.
When the discrepancy is significant even after the
above procedure, one can utilize the CI for settlement
of the problem. It is reasonable to decide that the,
decision with the lowest CI values is the most
consistent one. This is not to claim that the decision
with the lowest CIs is the best and thus should be
selected. This rule simply allows us to rank the
credibility of different decisions. The experts with
higher CIs may examine his/her matrices to find out
the reason for the large CI value. If the identified
reason is modifiable, the expert might wish to redefine
the relevant matrix elements. The discrepancy in the
expert decisions can be reduced through this
modification.
APPLICATIONS
The modified AHP method was applied to several
sample problems to test its usability. The sample
problems are a selection of options for the situations
described below:
Problem 1. Reduction of HEP in a transient
management operation.
Problem 2. Identification of the most influential
task for operability improvement.
Problem 3. Selection of maintenance policy.
The Problem 1 corresponds to the case illustrated in
Fig. 1. The transient assumed in this case is caused by
a feedwater controller failure. The Problem 2 is a
selection of one task out of a sequence of seven tasks
to be conducted for post-scram cooling down of a
nuclear power plant. The task should be selected in
terms of its possible contribution to operability
improvement after assumed modification. Here, the
term 'operability improvement' is evaluated with
multiple UIs shown in Fig. 2. The maintenance
policies considered in Problem 3 are characterized by
Choose lasks
to improve operability
II I I I
Llmilallon " Effect of IUnreliability] Iwork load I
on allowable time] a las~ failure
(Tasks)
Fig. 2. Hierarchical decision scheme of task selection for
operability improvement.

4. 70 M. Kitamura
I Selectionof MaintenancePolicy I
I
I I
Reliability I Cost
Attained Required
l
Average [ Minimum 1
Reliability | Reliability
I I 1
I
Fig. 3. Hierarchical decision scheme for maintenance
planning.
the maintenance interval and principle (i.e. preventive
or corrective). The decision structure of this problem
is illustrated in Fig. 3. More detailed descriptions of
these applications can be found elsewhere.4
The usability and advantage of the proposed
method were clearly confirmed in all problems. To
improve efficiency .of analysis, however, one addi-
tional modification was felt desirable. Instead of
identifying the difference in unstated assumptions
after AHP operation, articulation of assumptions
should be conducted at the stage of paired comparison
simultaneously. A formatted worksheet was designed
and introduced before analyzing the third problem.
The efficiency of the method was further improved
after this modification.
DISCUSSIONS AND CONCLUSION
The modified AHP method can contribute to reducing
the inter-expert discrepancy in PSA-related decision-
making. The basic idea, to decompose a complex
problem into a set of simpler problems organized in a
hierarchy, is quite simple yet effective. However,
other modifications are still needed for better
decision-making. Within the framework of the present
methodology, we need a tool for consistent record-
keeping of the modification of assumption. When the
modified assumption is influential, it calls for
redefinition of various matrix elements. Therefore, a
reliable method of record-keeping is of crucial
importance. Incorporation of a technique called Truth
Maintenance System (TMS) developed in the area of
artificial intelligences is currently being attempted to
meet this need.
Obviously, it is impossible to totally remove the
causes of inter-expert discrepancy by the proposed
method. However, the method clearly demonstrates
the usefulness of incorporating a knowledge-handling
technique to alleviate the technical difficulty in
PSA-based decision-making. Introduction of other
techniques of the similar discipline would be certainly
worthwhile.
REFERENCES
1. Watanabe, N., Kondo, M. & Abe, K., Japanese
Benchmark Exercise on Fault Tree Analysis. Paper
presented at IAEA International Symposium on the Use
of Probabilistic Safety Assessment for Operational
Safety, PSA'91, Vienna, 1991.
2. Saaty, T. L., The Analytic Hierarchy Process, McGraw-
Hill, New York, NY, 1980.
3. Saaty, T. L. & Alexander, J., Conflict Resolution, the
Analytic HierarchyApproach. Praeger, New York, 1989.
4. Washio, T., Kitamura, Y., Takahashi, H. & Kitamura,
M., Decision Support for Operability Improvement and
Maintenance Planning by Analytic Hierarchy Process
Methodology. In ed. Probabilistic Safety Assessment and
Management, ed. G. Apostolakis, vol. 2 Elsevier, New
York, 1991pp. 1451-6.
5. Doyle, J., A Truth Maintenance System. Artificial
Intelligence, 12 (1979) 273-9.