This slide-deck explains the concept of Data Driven Management (DDM) and it's application by CARE India in Bihar as a part of Integrated Family Health Initiative (IFHI) project funded by Bill & Melinda Gates Foundation. Later, in 2013, a much bigger, innovative and ambitious measurement effort called Concurrent Measurement and Learning (CML) was established as a part of Bihar Technical Support Program. The basic work on DDM during IFHI project created the basis for CML.

1. DATA DRIVEN MANAGEMENT
Overview and Application
A discussion on
How to Use Data in Management and Decision-
Making in Public Health Scenario in Bihar?
Suggestive slides for Visioning Workshop
Prepared by:
Indrajit Chaudhuri, CARE India
23rd July 2012

2. Indrajit Chaudhuri, 23rd July 2012
What is Data?
• Data is the value of different variables.
• Quantitative Data are generally represented by number or
percentage
• In the context of MCH, data on coverage, practices / behavior,
services provided etc. can provide understanding of
performance of the program. E.g.,
• No. of institution delivery (in a block / district)
• % of children received immunization
• No. of children breast-fed within one hour of delivery
• % of mothers who were visited at home by FLW thrice in first week after delivery
• No. of mothers received information on maternal complications by FLWs during last trimester
of pregnancy
Etc. etc.
• It is important to MEASURE / ASSESS to generate DATA

3. Indrajit Chaudhuri, 23rd July 2012
Why do we need data?
• For upward reporting:
– Calculation of cumulative national and sub-national estimates
– Planning
– Budget allocation
– Supports policy makers to develop policies and guidelines
• For decision making on the ground:
– Development of local-level strategies for implementation
– Targeting of issues on which performance is low
– Targeting of geographic areas or specific population groups
where indicators are poor

4. Indrajit Chaudhuri, 23rd July 2012

5. Indrajit Chaudhuri, 23rd July 2012
REASON FOR DATA
OFTEN NOT BEING USED
FOR DECISION MAKING
- specifically at the
implementation level
Information need for
managers at all level are
not assessed
Data collection decisions
are not made considering
the decision making need
at the implementation
level
Contextual data of
interest of Program
Managers are not
collected
Data is often not
available for the Program
Manager in a useable
form
Flow of data does not
ensure that it reaches
Program Managers at all
levels in a timely manner
And, also… Lack of
capacity to use the data
and realization of its
usefulness

6. Indrajit Chaudhuri, 23rd July 2012
What do we mean by Data Driven
Management?
• Data Driven Management (DDM) is the way of program management where
major decisions are taken on the basis of data.
• A Program Manager needs to take lots of decisions – day-to-day
implementation decisions, strategic decisions etc. – depending on the nature
of program and level of management.
• Any decision is taken on the basis of certain information. If the program
manager does not have those information – it may lead to wrong or imperfect
decisions.
• The Program Manager knows issues on which she needs to take decision. So,
she can identify well ahead what information she requires for taking those
decisions.
• In DDM, data is collected in order to provide those information to Program
Manager for facilitating the decision making process. Managers takes any
decision based on those data ensuring an objective decision-making process.

7. Indrajit Chaudhuri, 23rd July 2012
How Data Driven Management
works?
• Data Driven Management follows following few steps:
– Identification of information requirement:
– What all information do we need to manage the program and take
important decisions (at various levels) ?
– Preparing strategy for capturing those information:
– What data should be collected for making those information
available in a timely manner? How should those data be collected?
How should data flow?
– Analyzing Data in order to make relevant information available:
– How should the data be analyzed? How can the analysis of data be
presented in a usable form, which provide timely, optimal and
required information for decision-making?
– Using analyzed data for taking important program decisions –
mainly in terms of evaluating progress and setting future target

8. Indrajit Chaudhuri, 23rd July 2012
THE DATA DRIVEN MANAGEMENT
FRAMEWORK
MEASURE
IDENTIFY
GAPS
STRATEGIZE
TAKE
ACTION
Measure output /
outcome / impact
level indicator TO
GENERATE DATA
ANALYSE DATA to
identify low
performing areas
(SC / blocks etc.) &
reasons for low
performance
Prepare / modify
strategies and
prepare plans for
particular
geographic area
Measure again !
Take
appropriate
action as
per strategy
SET A TARGET – after
measuring each time as
reference for the next
assessment

9. Indrajit Chaudhuri, 23rd July 2012
Home visit by
FLW at right
time
Delivery of
appropriate
messages and
effective
counseling
Change in
behavior
Measure
Identify
Gaps
Strategize
Take
Action
Measure
Identify
Gaps
Strategize
Take
Action
Measure
Identify
Gaps
Strategize
Take
Action
Application of DDM Framework
An example of application of Data Driven Management Framework
in improving behavioral outcomes by applying it at for intermediary
outputs responsible for the final outcome

10. Indrajit Chaudhuri, 23rd July 2012
MEASURE
VARIOUS INDICATORS COULD BE MEASURED TO GENERATE DATA
Measuring the final impact is important. But, the changes in impact-level
indicators depend on changes in many smaller actions, which could be
measured through various output / outcome level indicators. Therefore, for
program managers, it is important to measure intermediary outcomes and
outputs – in order to identify the gaps clearly.
An example is used for describing this in the next slides…. In the example
first week PNC visits are taken as example.

11. Indrajit Chaudhuri, 23rd July 2012
FINALLY…
MEASURE
IMPACT
MEASURE
OUTCOMES
MEASURE
FREQUENCY
OF HOME
VISIT –
WHETHER
FLW VISITED?
WHETHER MESSAGES
WERE DELIVERED?
Home Visit by
FLWs during
the first week
of delivery Delivered message
on “nothing to be
applied on the
cord”
Clean cord care
practiced
Delivered message
on “skin to skin
care”
Thermal care
practiced
Delivered message
on “only breast
milk”
EBF practiced
Delivered message
on maternal
danger signs
Recognized &
treated of
maternal
complication
Reduced
Maternal
Mortality
Delivered message
on neonatal
danger signs
Recognized &
treated of
neonatal
complication
Reduced
Neonatal
Mortality
EXAMPLE

12. Indrajit Chaudhuri, 23rd July 2012
IDENTIFY GAPS
DATA COULD BE ANALYZED TO IDENTIFY GAPS
After measurement of indicators at various output and outcome level –
generated data are analyzed in order to identify gaps.
It is important to identify gaps in terms of:
• Where in the chain of output and intermediary outcomes is there a drop
in performance?
• Finding out specifically low performing regions: Blocks, Sub-centers or
catchment areas etc. which are not performing below the acceptable
standard
• Finding out socio-economic groups – where performance of some
indicators are low

13. Indrajit Chaudhuri, 23rd July 2012
EXAMPLE
ImpactOutcome 2Outcome 1
Output X
Action A
Action B
Output YAction C
Where in the chain is there a drop in
performance or achievement?
Which geographical areas are
low performing?
(with respect to a particular
indicator)

14. Indrajit Chaudhuri, 23rd July 2012
STRATEGIZE
STRATEGIES AND PLANS ARE PREPARED FOR MITIGATING THOSE GAPS
Data driven management helps in sharp identification of gaps – which
helps in building the strategy.
The preparation of strategy should consider the TARGET that is set after
measurement, which is specific to the indicator in the particular area.

15. Indrajit Chaudhuri, 23rd July 2012
TAKE ACTION
APPROPIATE ACTION SHOULD BE TAKEN AS PER STRATEGY
After the action is taken, measurement should be repeated. There should
be an agreed newly-set target and agreed time-frame within which the
changed strategy should reflect in the measurement.
If the measurement reveals large gap from the target – the process of gap
finding and re-strategizing should be repeated.

16. Indrajit Chaudhuri, 23rd July 2012
In the first three
quarters home visit
increased, but
advices did not
improve
0
20
40
60
80
Q1 Q2 Q3 Q4 Q5 Q6
% of women received 3
PNC visits by FLW
% of women received
any advice from FLW
% of women received
all advices from FLW
EXAMPLE PNC visit data of a particular area for consecutive quarters
Data is
continuously
measured and
analyzed in terms
of visit and advice
by FLWs
MEASURE &
ANALYSE

17. Indrajit Chaudhuri, 23rd July 2012
In the first three
quarters home visit
increased, but
advices did not
improve
Gap
identification
exercises show
poor content
delivery because
of lack of
capacity of FLWs
EXAMPLE PNC visit data of a particular area for consecutive quarters
Analysis of data of
visit and advice by
FLWs helps in
identifying gaps
0
20
40
60
80
Q1 Q2 Q3 Q4 Q5 Q6
% of women received 3
PNC visits by FLW
% of women received
any advice from FLW
% of women received
all advices from FLW
IDENTIFY
GAPS

18. Indrajit Chaudhuri, 23rd July 2012
0
20
40
60
80
Q1 Q2 Q3 Q4 Q5 Q6
% of women received 3
PNC visits by FLW
% of women received
any advice from FLW
% of women received
all advices from FLW
EXAMPLE PNC visit data of a particular area for consecutive quarters
Identification of
gaps help in
preparing better
strategies and plan
for improvement of
outcome
In the first three
quarters home visit
increased, but
advices did not
improve
Gap
identification
exercises show
poor content
delivery because
of lack of
capacity of FLWs
STRATEGIZE
Increased
emphasis on
content
delivery

19. Indrajit Chaudhuri, 23rd July 2012
0
20
40
60
80
Q1 Q2 Q3 Q4 Q5 Q6
% of women received 3
PNC visits by FLW
% of women received
any advice from FLW
% of women received
all advices from FLW
EXAMPLE PNC visit data of a particular area for consecutive quarters
Action taken as per
revised strategy
helps in improved
outcome
In the first three
quarters home visit
increased, but
advices did not
improve
TAKE ACTION
& MEASURE
AGAIN
Gap
identification
exercises show
poor content
delivery because
of lack of
capacity of FLWs
Increased
emphasis on
content
delivery
0
20
40
60
80
Q1 Q2 Q3 Q4 Q5 Q6
% of women received 3
PNC visits by FLW
% of women received
any advice from FLW
% of women received
all advices from FLW

20. Indrajit Chaudhuri, 23rd July 2012
APPLICATION OF
DATA DRIVEN MANAGEMENT
IN OUR CONTEXT
– IN THE CONTEXT OF PUBLIC HEALTH IN BIHAR

21. Indrajit Chaudhuri, 23rd July 2012
CURRENT STATUS
WITH RESPECT TO DATA IN HEALTH SECTOR IN BIHAR
• The value of data has immensely increased in recent years. Lots of data
are being collected. They are being used for reporting above, planning and
setting overall target and budget. But use of data on taking management
decisions at the implementation level is very limited.
• Available Data Sources: HMIS, different large population surveys (NFHS,
DLHS etc.)
• Issues with available data
o HMIS: Self-reported – possibility of errors; Meant for upward
reporting
o Large population Surveys: Data for smaller geographic areas are
not available; Data are available after long time – loss relevance.
o Mainly coverage / final outcome data – data on intermediary steps
are mostly not available
• Therefore, all these available data are generally not used for day-to-day
decision making

22. Indrajit Chaudhuri, 23rd July 2012
HOW CAN WE RESOLVE THIS ISSUE?
WHAT DO WE NEED FOR EFFECTIVE DDM?
• Population level surveys are best ways to measure output and outcome
level indicators.
• Random sample ensures unbiased estimate.
• Data flow should be fast – in order to have minimum time lag between
data capture, analysis and use – so as to generate almost real-time
estimates.
• So, a population level random sample survey covering all the important
output and outcome level indicators with capability of generating real-
time data can help in initiating the Data-driven Management on the
ground.
A probable solution, which can work well for the block and
district level, could be employing LQAS methodology – with
real-time data transfer and analysis mechanism.

23. Indrajit Chaudhuri, 23rd July 2012
WHAT IS LQAS?
• Lot quality assurance sampling (LQAS) is a random sampling
methodology, that helps us generate an understanding of performance /
achievement in a supervisory area (e.g., block) with very small number of
random sample.
• In our context there can be two way use of LQAS:
– It can be used at the block-level to identify ‘priority blocks’ or ‘priority
indicators in a block’ – which are not achieving the target or an
established benchmark
– It can also provide a measure of coverage or estimate of various
indicators at the district-level
• The beauty of employing LQAS is that it can work effectively with a very
small sample size at the block-level – which is as small as 19 – when
randomness of the sample is ensured.

24. Indrajit Chaudhuri, 23rd July 2012
• A small number of random sample is selected from each block.
– 19 is the most common and most efficient sample size for LQAS
– These samples are checked for any specific indicator which can be
binomially expressed (like – “yes/no”, “achieved/not achieved”,
“received/not received” etc.).
• A target (expressed by the term ‘decision rule’) is pre-set to indicate the
accepted result in that indicator at the block level. The ‘decision rule’
indicates number of respondents from sample that should be found to
meet the criteria for that indicator (Decision rule is determined from the
table shown in a slide later).
– If the pre-set target for an indicator is 80%, then the LQAS table shows
that the decision rule should be 13 out of 19 samples. This means: if
less than 13 samples of a block meet the criteria for an indicator, then
the target for that block is not achieved.
HOW IS LQAS USED?

25. Indrajit Chaudhuri, 23rd July 2012
How to interpret LQAS data?
• At the district level: District estimates are available with fair
precision.
• At the block level:
– We do not get any coverage estimate, but, we get to know
• which of the blocks do not meet the “target” in a particular
indicator
• which particular indicators did not meet the “target” in a
particular block
– The best use of LQAS (at the block-level) is to find under-
performing blocks and underperforming indicators in a block.
– A TARGET SHOULD BE PRE-SET FOR GENERATING THESE
ESTIMATES AT THE BLOCK-LEVEL.

26. Indrajit Chaudhuri, 23rd July 2012
FROM OUR LQAS - ROUND 1
IFHI has already undertaken LQAS between the months of December and
February. The method is operationally tested now.
IFHI block coordinators collected data from 19 mothers of each of the 4 age
groups of children (0-2 months, 3-5 months, 6-8 months and 9-11 months).
The data was captured also through hand-held devices and real-time estimates
and analyses were available for use at the block and district level.
Results of the round-1 are expressed in next few slides. As targets were not set
beforehand in the block with block-level managers – we are using dummy
targets for analysis.

27. Indrajit Chaudhuri, 23rd July 2012
PLEASE INSERT RELEVANT SLIDES
FROM YOUR “LQAS DISTRICT
RESULT” PRESENTATIONS
(which were shared earlier)

28. Indrajit Chaudhuri, 23rd July 2012
TARGET SETTING
– LET US SET A TARGET FOR THE NEXT ROUND ON A
FEW SIMPLE INDICATORS
for the district (overall)
&
for a block (if necessary)

29. Indrajit Chaudhuri, 23rd July 2012
HOW TO SET TARGETS?
• We can select some indicators which we feel will be improved in next three
month. Say, indicators on home visit and delivery of some contents through
FLW interaction (say, on BP).
• We can get the district estimate and a rough idea about block situation
(from color – red/green and indicative number) from Round 1. We can also
get an estimate of various indicators from the Ananya Baseline data.
• We can discuss about these few indicators in district-level visioning
workshops to set a overall district target for the next three months.
• We can also discuss about these indicators in visioning workshops at the
block-level to agree with them on the same target
– If some of the blocks feel that the target set at the district level is too
high and not contextual for their district revision of the target for the
particular block could be done.
• We can discuss these targets with FLWs in ANM Tuesday meetings, ASHA
divas meetings and AWW monthly meetings to get their ownership.
• Then, after the next round we can see whether blocks met those targets or
not (from the LQAS decision rule table).

30. Indrajit Chaudhuri, 23rd July 2012
HOW KEEP OUR FOCUS & MONITOR
THESE TARGETS DURING THE QUARTER?
• LQAS data will be available after the quarter. But it is important to keep
the attention of FLWs on these targets. There are few possible ways to do
that:
• These few indicators should be discussed in all possible forums
with FLWs to keep the attention of FLWs maintained. The
discussion with FLWs can happen in Sub-center Platform Meetings,
ANM Tuesday meetings, ASHA divas meetings and AWW monthly
meetings.
• Some of these measures may be available from HMIS or some
other data source of IFHI. These data should be analyzed and
presented to FLWs on a monthly basis to keep a track on the
progress.
• Information available from Home Visit Registers should be
discussed in reference to these indicators in all the monthly sub-
center meetings.
Etc.

31. ANNEXURES
SLIDES TO BE USED FOR
EXPLAINING CERTAIN CONCEPTS –
IF NECESSARY

32. LQAS Decision Table
Taken from A Participant’s Manual, Joseph J Valadez et al.

33. Indrajit Chaudhuri, 23rd July 2012
Why use a Sample Size of 19?
• Little is added to the
precision of the
measure by using a
sample larger than 19.
• Sample sizes less than
19, however, see a
rapid deterioration in
the precision of the
measure.
Sample size

34. Indrajit Chaudhuri, 23rd July 2012
What do we need to remember from the
decision table?
– For 19 samples:
– Therefore, for a sample size of 19 per block, if the target is 50%
for an indicator (say, initiation of breast-feeding within one hour
of delivery), then all the blocks, which had less than 7 samples
as ‘yes’ (i.e., if less than 7 women out of 19 found initiated
breast-feeding within one hour) will be identified as ‘not met
the target’ and will be marked in “Red”.
– Target can be set by the implementation team (generally, district
team) and the decision rule will change accordingly. E.g., if
target is 60%, decision rule will be 9, 11 for 70% and 13 for 80%.
Target 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
Decision
Rule
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

35. Indrajit Chaudhuri, 23rd July 2012
USE OF DATA DRIVEN MANAGEMENT IN
CONTEXT OF ULTIMATE VISION OF MCH
• Ultimate vision of MCH is to reduce IMR, Malnourishment, TFR and MMR
• But, it is difficult to measure these. We can measure proximal outcome
indicators – which can indicate whether we are in right direction.
• In order to see whether our program is in right direction to reduce IMR – we
should find out:
• Whether identification of newborn complications are increasing
• Whether more newborns are getting treatment for complications
• Whether FLWs are providing right message and right counseling regarding newborn
complications at right time through home visit Etc.
• In order to see whether our program is in right direction to reduce
Malnourishment – we should find out:
• Whether exclusive breast feeding rates (till six-month) are increasing
• Whether age-appropriate frequency and quantity of complementary feeding is
increasing with continuation of breast-feeding from six-month age of the child
• Whether initiation of complementary feeding at the age of six month is increasing.
Etc.
CONTINUED…

36. Indrajit Chaudhuri, 23rd July 2012
USE OF DATA DRIVEN MANAGEMENT IN
CONTEXT OF ULTIMATE VISION OF MCH (Continued)
• In order to see whether our program is in right direction to reduce TFR –
we should find out:
• Whether unmet need for contraception is decreasing
• Whether
• Whether more newborns are getting treatment for complications
• Whether FLWs are providing right message and right counseling regarding newborn
complications at right time through home visit
• In order to see whether our program is in right direction to reduce MMR –
we should find out:
• Whether identification of maternal complications are increasing
• Whether more mothers are getting treatment for complications
• Whether FLWs are providing right message and right counseling regarding maternal
complications at right time through home visit Etc.
• Thus Data Driven Management provides us much easier and simpler ways
to understand the progress towards achievement of these high-level
indicators like MMR, IMR, TFR and malnourishment.