By Catherine R. Ragasa, Aliyu Sabi Abdullahi, and George Owusu Essegbey. Presented at the ASTI-FARA conference Agricultural R&D: Investing in Africa's Future: Analyzing Trends, Challenges, and Opportunities - Accra, Ghana on December 5-7, 2011. http://www.asti.cgiar.org/2011conf

1. Measuring R&D Performance within
Innovation Systems Perspective:
Illustrations from Ghana and Nigeria Agricultural Research
Systems
Catherine Ragasa (IFPRI)
George Essegbey (STEPRI)
Aliyu Sabi Abdullahi (ARCN)

2. Motivation of the study
• Investment in R&D (usually coupled with infrastructure)
is required for increasing agricultural productivity
• High ROR on investment on R&D (although with wide
variability)
• Increased investment emphasized, but there is dearth in
studies on the functioning of NARIs/NARS where the
intended funding would go to
• Are agricultural research organizations within Nigeria and
within Ghana performing differently and what are the factors
that account for these differences?
• Are there patterns of differences between Ghana and
Nigeria agricultural research performance measures that are
explained by differences in their policies, emphasis and
practices?

3. Measures of performance
• Conventional indicators
• Number of publication (self-reported)
• Number of technologies (self-reported)
• Number of patents (self-reported)
• Organization theory and public sector motivation literature
• Staff morale (staff perception; Likert scale)
• Peer rating (perception of other organization; frequency)
• Number of awards (self-reported)
• Innovation systems perspective
• Interaction with various actors (frequency, perception of usefulness)
• Involvement of actors in priority-setting; use of participatory approaches and
innovation platforms (dummy)
• Level dissemination efforts of publications and technology (self-reported;
number)
• Extent of use of publications (staff perception; Likert scale)
• Extent of adoption of technologies (staff perception; Likert scale)

4. Potential explanatory factors
• Organizational capacity (satisfaction on adequacy; actual ASTI data)
• Human, physical, financial resources
• Management systems (satisfaction on adequacy/implementation)
• Planning processes
• Information and coordination processes
• Monitoring and evaluation system
• Perception of transparency and other dimension of work environment
• Training (technical and management)
• Organizational culture types – dilemma of control versus flexibility
and people versus organization
• Group, development, rational, hierarchical, balanced culture types
• Incentive systems & sources of accountability
• Performance indicators reported being used (open-ended question)
• Rewards/sanctions being used (open-ended question)
• Salary (Salary costs/FTE; staff perception)
• Openness of information about organization (staff perception, Likert scale)

5. Method and Data Sources
• NARIs under ARCN and CSIR, CRIG, and HEI
• Nigeria
• 47 organizations [all 15 RI, all 11 FCA, 21/48 faculties]
• 344 randomly-selected researchers (out of >3,000 total) [3-20
sample/organization]
• Ghana
• 16 organizations [all 10 RI; CRIG; and 6/15 faculties]
• 237 randomly-selected researchers (out of 706 total) [5-20
sample/organization]
• Face-to-face survey
• 2 types of questionnaire (for head and for sample researcher)

6. Majority of technologies produced are improved
management practices
100% 2
19 10
90%
80%
56
70%
60% 59
Chemical
50% Mechanical
72
40% Management
30% Biological
20%
38
58
10%
0%
Nigeria (205) Ghana (109) * Preliminary

7. Limited awareness of adoption of technologies in both
Ghana and Nigeria
100% 1
90% 13 15
Wide adoption (>60%)
80% 4
5
70% 29 Moderate adoption (41-
60%)
60%
Some adoption (21-40%)
50% 17
40% Limited adoption (<20%)
3 76
30%
Zero adoption
20% 37
10% No knowledge
0%
Nigeria (87) Ghana (114)

8. Observation (1)
• Lack of farmer- or impact-orientation; mainly supply-driven
• Heavy emphasis on publication and technology (N) and heavy
emphasis of internally-generated funds (G); little follow-up on what
happens to the technologies
• Use of indicators such as “impact on/adoption of farmers”,
“farmers’ problems solved”, “appreciation by farmers,” as
performance indicators
• 0% (G); 30% (N)
• Organization heads who mentioned these as motivations
• 0% (G); 20% (N)
• Researchers who mentioned these as motivations
• 15% (G); 5% (N)
• Organization heads who mentioned these or development
outcomes (e.g., food security, poverty reduction) as central mission
of organization according to org head
• 20% (G,N)

9. What motivates researchers?
Nigeria Ghana
50 60
45
40 50
35
30 40
25
30
20
15 20
10
5 10
-
-

10. What motivates organization heads?
Nigeria Ghana
70 60
60 50
50
40
40
30
30
20
20
10
10
0
0

11. Observation (2)
• Different motivations across organizations and countries
• Most of the motivations are already being implemented
(promotion, recognition, higher salary or bonus based on
performance, training), but need more transparency in
promotion and training participant selection; better
implementation of M&E
• What are not being addressed are the basic needs (such as
basic lab and research facilities, electricity, and internet).
Conducive work environment has mainly to do with adequacy
of resources.
• More timely research funds and research funds as motivating
factors for both researchers and org leader
• Explore using other indicators not yet explored – appreciation
by farmers

12. Observation (3)
• Nigeria has more publications per researcher
• Nigeria - stronger emphasis on publication (in both RI and HEI) in
explicit performance indicators used and implicit measures
recognized by peers.
• Ghana – more focus on internationally-generated funds and
associated more frequency of consultancy work rather than peer-
reviewed publications
Average count of publications in last 3 years per
8
researcher
6
4 Nigeria
2 Ghana
0
Total With BS With MS With PhD

13. Limited linkages among various actors
Nigeria Ghana
100% 100%
90% 90%
Several times a
80% 80% month
70% 70%
About once a
60% 60% month
50% 50%
A few times a year
40% 40%
30% 30%
Never
20% 20%
10% 10%
0%
0%

15. Observation (4)
• Linkages are important (staff, org head, correlation
analyses)
• 2 strategies: (a) changing mindset; and (b) addressing the
binding constraints that hinder greater linkages, which include
lack of funds, platforms, and transportation, time constraints, no
incentive and no culture in the organization, or lack of interest or
motivation from both parties
• Researchers in Ghana reported having better linkages with
other innovation actors, but researchers in Nigeria reported
having better linkages with other researchers.
• Ghana - greater emphasis on non-state funds generated which
would very much require interaction and collaboration with
possible sources of funds, such as private sector, NGOs, and
international organizations.
• Nigeria – greater emphasis on publication, often requiring
greater interaction with other researchers

16. Observation (5)
• Variability in organizations within Ghana and Nigeria
• Good performing and bad performing organizations in the same country
or institutional context; elements of good performance
• Factors (correlation analyses)
• Greater satisfaction of facilities/physical resources more technology
produced per PhD, more awards received, peer rating (N); more
technology and publication per PhD (N, G)
• International research collaboration more technologies, more
publication, better perceived adoption, award, peer rating (G, N)
• Higher PhD/MS ratio higher publication per MS (G)
• Higher operating funds per FTE more publication per MS (G)
• More frequent linkages with farmers, private sector, MOA, other
researcher, Intl Org more technologies & publication, award, peer rating
(G,N)
• M&E system, work environment, international research collaboration (N);
control (G) more researchers perceiving more adoption

17. Observation (6)
• Work environment almost all performance indicators
• Majority of staff are satisfied with their job
• Majority reported that their organizations are effective given its
budget and resources (Ghana >Nigeria)
• Most important issue/dissatisfaction in both countries is
adequacy of physical resources and research funds
• The second items that show much dissatisfaction among
respondents:
• Not-so competitive salary and benefits seem to be a bigger
problem by staff in Ghana; while job security seems to be a
much bigger problem in Nigeria.
• There is also reported dissatisfaction because of the level of
corruption or misuse of funds in the organization and
transparency in promotion and hiring processes for both
countries (more in Nigeria)

18. Observation (7)
• Output and productivity seems to respond to performance
indicators
• If publication is mentioned as a performance indicator more publications
being produced by its researchers.
• If technology is mentioned as a performance indicator more technologies
that researchers will contribute in producing.
• Presence of reward or sanction more technologies and
publication per PhD and per MS
• Openness of information about performance of organization
is correlated to publication, technology and peer rating
• Salary & benefits have been consistency mentioned, but
variations of salary (variations in perception on competitive
pay, adequacy of salary in relation to living expenses, and
salary costs per FTE did not seem to be statistically
correlated with variations in any of the performance
indictors)

19. Observation (8)
• Different performance ratings can be inconsistent or contradictory.
Puzzles:
• More researchers in Ghana seem to have more linkages, more satisfied with
human and physical resources, organizational management practices, and work
environment, but these do not seem to translate into greater productivity
(publication, technology)
• More researchers in Ghana perceived that their organizations are effective given
its budget and resources than in Nigeria
• Within Nigeria - more perceived adoption, award & peer rating all are positively
correlated; but all are negatively correlated with productivity (publications and
technology per PhD and per MS)
• Within Ghana – productivity and peer rating are positively correlated but all are
negatively correlated with perceived adoption level
• From analysis perspective, avoid using composite index to capture
all performance indicator, but to analyze them individually
• Trade-offs or balanced set of indicators? Trade-off between or
combination of scholarly contribution versus impact of poor
farmers and end-users

20. Concluding remarks
• Complex and challenging process
• We tried a participatory approach, engaging researchers in
several iterations of the questionnaire (technology, adoption,
can mean differently to different people)
• Revisit definitions and measures especially when these
metrics are to be scaled out to other countries
• Self-reporting (CV, not an evaluation, capacity strengthening)
• Time lag of technologies and attribution to various factors
• This study is a starting point; and more needs to be done
• Qualitative approach (in-depth case studies with observational
approaches and historical perspective)
• More research is needed to look at actual adoption and
impact of technologies (beyond perceptions of scientists)

21. Concluding remarks
• Funding is required for basic institutional capacity
• Infrastructure seems to be a binding constraints
• Linkages are important (but not automatic); linkage costs can be explicitly put
in the budget
• Salary & benefit improvements are consistency emphasized by researchers
and leaders, but needs to be further studied
• Human capacity strengthening (technical, management, value chain
approaches)
• How can we make NARIs be more responsive and accountable to the needs
and demands farmers?
• A lot of it is leadership/management commitment
• Performance indicators/targets, coupled with transparency
• Openness of information about org performance is associated with greater
productivity and better peer rating
• Hosting producer organizations within RI helps in the linkages
• External accountability
• Greater international research collaboration
• Producer organizations managing competitive grants