SlideShare une entreprise Scribd logo
1  sur  97
Télécharger pour lire hors ligne
Framework Contract EuropeAid: 127054/C/SER/multi - Lot 1
Rural Development
Evaluation of the Somalia Food Security and
Nutrition Analysis Unit (FSNAU)
Reference Number 2012/285-027/1
DRAFT - FINAL REPORT
April 2012
Report submitted by
LANDELL MILLS LTD, in Consortium with VetEffecT
This report has been prepared at the request and with the financial support of the European
Commission. The content of this report is the sole responsibility of Landell Mills and/or
VetEffecT and can in no way be taken to reflect the views of the European Commission.
KEY DATA
Name of Project: Evaluation of the Somalia Food Security and Nutrition
Analysis Unit (FSNAU)
Contractor: DEU to Kenya, Somalia Unit
Contracting Authority: 18/03/2012 – 15/06/2012
Start/End Date: Somalia Food Security and Nutrition Analysis Unit (FSNAU)
Beneficiary: Nairobi, Kenya
Primary Location: Somaliland, Puntland and rest of Somalia
Secondary Locations: DEU to Kenya, Somalia Unit
DISTRIBUTION LIST
Recipient Copies Format
Somalia Unit, Delegation of the European Union to the
republic of Kenya
10 Hard copy
Somalia Unit, Delegation of the European Union to the
republic of Kenya
1 electronic copy in
PDF
Somalia Unit, Delegation of the European Union to the
republic of Kenya
1 electronic copy in
word
ACKNOWLEDGEMENTS
The authors wish to thank the EU delegation and stakeholders for the good collaboration for
the implementation of the project.
Page i
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012 Apr
ABBREVIATIONS AND ACRONYMS
AGN the nutrition and consumer protection division at FAO
CAP (Inter-agency) Consolidated Appeal Process (UN)
Candlelight Candlelight for Health, Education & Environment
CDC Centre for Disease Control
CHF Common Humanitarian Fund
CS Central and South Somalia
DAC Development Assistance Committee: criteria for evaluation of development assistance of
the OECD
DEU Delegation of the European Union
DFID Department for International Development (UK)
DIAL Development Initiative Access Link
EC European Commission
ENA Emergency Nutrition Assessment
EQ Evaluation Question
ESAF The Food Security and Agricultural Projects Analysis Unit of FAO
EU European Union
EWS Early warning System
FAO Food and Agriculture Organisation
FEWS-Net Food and Early Warning System Network
FNS Food and Nutrition Security
FS Food Security
FSAU Food Security Analysis Unit (1995 – 2009)
FSNAU Food Security and Nutrition Analysis Unit (2009 - present)
GAM Global Acute Malnutrition
HADMA Humanitarian Affairs and Disaster Management Agency of Puntland
HAVOYOCO Horn of Africa Voluntary Youth Committee, Somaliland based NGO
HDI Human Development Index
IASC Inter-agency Standing Committee
ICP Integrated Phase Classification
ICRC International Committee of the Red Cross
IDP Internally Displace People
IEC International Education Center
IGAD Intergovernmental Authority on Development
INGO International Non-Government Organisation
IPC Integrated Phase Classification
JRC Joint Research Centre of the EC
KAP Knowledge, Attitudes, Practices
LNGO Local Non-Government Organisation
LRRD Linking Relief, Rehabilitation and Development
NE North East Somalia or Puntland
NERAD National Environmental Research and Disaster Commission of Somaliland
NGO Non-Government Organisation
NIPHORN Nutrition Information Project for the Horn of Africa
NW North West Somalia or Somaliland
OCHA Office for the Coordination of Humanitarian Affairs
OECD Organisation for Economic Co-operation and Development
Oxfam-GB Oxford people for famine Great Britain
PENHA Pastoral and Environmental Network in the Horn of Africa
PMAC Project Management Advisory Committee (Nairobi based)
PoA Plan of Action (FAO)
PTF Project Task Force (FAO Rome based)
PRRO Protracted Relief and Recovery Operation (WFP)
SAM Severe Acute Malnutrition
SCF Save the Children Fund
SDC Swiss Agency for Development and Cooperation
SIDA Swedish International Development Agency
SLIMS Somali Livelihood Indicator Monitoring System
SMART Standardized Monitoring and Assessment of Transitional Relief
Page ii
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012 Apr
SSS Somali Support Secretariat
SWALIM Somalia Water and Land Information Management
TFG Transitional Federal Government
TL Team Leader
TM Team Member
ToR Terms of Reference
TTF Technical Task Force
UN United Nations
UNDP United Nations Development Programme
UNHCR United Nations High Commissioner for Refugees
UNICEF United Nations Children’s Fund
UNSCN United Nations Standing Committee on Nutrition
USAID United States Agency for International Cooperation
USD United States Dollars
VAM Vulnerability and Mapping
VETAID Somaliland based NGO part of East Africa Veterinary Aid NGO network
WASH Water, sanitation and hygiene
WFP World Food Programme
WHO World Health Organisation
Page iii
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012
Apr
CONTENTS
ABBREVIATIONS AND ACRONYMS .............................................................................................I
EXECUTIVE SUMMARY ................................................................................................................1
1. INTRODUCTION..................................................................................................................7
2. FINDINGS..........................................................................................................................21
3. CONCLUSIONS AND RECOMMENDATIONS ..................................................................33
Page iv
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012
Apr
Page 1
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
EXECUTIVE SUMMARY
1. Purpose of the Evaluation
The objective of the evaluation of phase VI (2009-2013) of the FSNAU is to provide an
assessment of the relevance, appropriateness, efficiency, impact and coherence of the
FSNAU programme with a view to informing future direction for phase VII (2013-2017).
Key questions to be answered are:
1. To what extent has food security and nutrition information and analysis produced by
FSNAU been used and for what purposes?
2. To what extent are FSNAU activities appropriate and tailored to local conditions and
needs? Are they carried out taking longer-term development and humanitarian
scenarios into account?
3. Has FSNAU and FAO created synergies between information system projects of
FSNAU and SWALIM?
4. To what extent has FSNAU contributed to improved knowledge on acute and chronic
food insecurity and malnutrition and their root causes in Somalia.
The evaluation also responds to specific questions that other donors or clients might have.
Chapter 2 gives a general introduction about the overall context of the situation in Somalia
and the food and nutrition situation in particular. It also explains how the evaluation has been
carried out, the methodology used and the formulation of the evaluation questions. The
evaluation attempts to give an answer to the four key evaluation questions (EQ 1-4) listed in
the Terms of Reference (ToR), the specific questions asked by the FSNAU management
(EQ 5-8), the one specific additional question formulated by the DEU Somalia unit (EQ 9)
and the specific DAC evaluation criteria (EQ 10-23). In chapter four and five conclusions are
drawn and recommendations are given.
The evaluators have interviewed the various clients of FSNAU (see annex 2): Somalia
government agencies, national NGOs, international NGOs, UN agencies and technical
clusters but also all donors. This was done according to a list of specific questions that
reflect the specific demands formulated in the ToR (chapter 2.2).
2. Main Analytical Points and Findings
1. The FSNAU provides mainly information of high quality used by a great number of clients
varying from local NGOs in Somalia to major donors. This information is used by all
clients for various purposes but mainly for donors to justify funding of programmes, for
UN agencies and international NGOs to design interventions, and for national NGOs and
governmental agencies in the different parts of Somalia – Somaliland, Puntland and CS
Somalia – to design specific policies and interventions etc. In general the information is
found to be useful by the clients.
2. However, a number of clients mention that the enormous quantity and the complexity of
the information make it difficult to find specific information or to understand the
information. This has to do with the way information is structured and as such is
insufficiently well tailored for specific clients. FSNAU produces a large quantity of
information that is understandable for the experts in food security and nutrition but not for
NGO or government field workers, general programme managers etc.
3. Existing debates about the concepts of “food security and nutrition” or “food and nutrition
security” and the way FSNAU collects and processes all data, refer in the first place to
Page 2
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
FSNAU history. FSNAU was created in 2009 from merging FSAU and the Nutrition unit.
The fact that FSNAU has a separate Food Security section and a Nutrition section has
not been appreciated by all concerned. Concerns exist such as “FSNAU is too FAO”,
“FSNAU has a monopoly on data collection and interpretation”, FSNAU is a “Ministry”
etc. FSNAU is very aware of these concerns and tries to be as transparent as possible.
The question of concepts of “food security and nutrition” or “food and nutrition security” is
quite academic. FSNAU collects both food security and nutrition data; nutrition data play
an important role in the process of validation. This question needs more “development”
and FSNAU is very aware of the need to clarify its role. Optimal client participation in this
development should therefore be an option to improve the realisation of FSNAU’s
general objective and purpose.
4. A major concern is how to transfer FSNAU tasks to the Somali government as much as
possible in an efficient but also effective way given the political situation, security and
level of capacities of existing national governmental agencies. Transfer means more
sustainability but on the other hand less efficiency and effectiveness given the high
turnover of qualified staff in the governmental services etc.
5. FSNAU contributed to improved knowledge on acute and chronic food insecurity and
malnutrition but the understanding of their root causes can still be improved. In general it
is understood that both forms of food and nutrition insecurity are mainly caused by food
but also by water, sanitation and hygiene (WASH) and health related factors but to what
degree is not known. To improve understanding, the Health and the WASH clusters
request inclusion of more WASH and health related indicators of food and nutrition
insecurity. FSNAU is very aware of this concern but indicates that a distinction must be
made between “immediate” and “underlying” causes of food insecurity and malnutrition.
6. The question of synergies between the SWALIM and FSNAU information system
projects is relevant. The ideal situation should be the full integration of SWALIM and
FSNAU information systems with different sections that horizontally cover a number of
sections related to food security, food production, livestock, markets, water, hydraulics,
irrigation, nutrition etc. and vertically the specific needs for emergency, Linking Relief,
Rehabilitation and Development (LRRD) and development information. The specific
existing areas of synergies now provide complementary information on water, climate
and ecology etc. with specific distinction of in-household water availability and extra-
household water availability.
7. The question about whether FSNAU should continue to be the institution that determines
the emergency or no-emergency situation in Somalia is understandable but not realistic.
There is no alternative for information provision. National governmental agencies do not
have the capacity to build an information system such as FSNAU. The perspective to
build them up will take many years and certainly higher financial investments. The
security and political situation in Somalia is not improving. The arguments used are very
reasonable but the reality is that no alternatives are available. Finally it must be
recognised that a lot has been achieved under FSNAU and developing a better
alternative is not realistic at the present time.
8. Besides the way the FSNAU is organised in sections – food security, nutrition, livelihood
– and the way IPC develops its methodology there is a certain concern about the way
the workload is divided. Heads of section spend a substantial part of their time on editing
the many reports. There is a great need for a professional editor to help the project to
restructure the stream of information. The heads of section’s main duty is not editing
reports but management of sections and coaching of employees etc.
Page 3
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
3. Main Conclusions
I. Use of information?
Information produced by FSNAU is used in an optimal way by all clients mainly for project
design, justification of new funding, filling the annual Consolidated Appeals, policy
development and follow-up of trends. It is also used to declare the situation of emergency or
not emergency.
II. Appropriateness and if enough tailored to local conditions and needs?
As long as the provided information is clear and tailored enough the appropriateness of the
provided information is not questioned. However, a number of clients express their concerns
about the quantity of information and the high technical level in which it is published. This
indicates that despite an appropriate assessment methodology, the information is not
sufficiently tailored to local conditions and needs.
III. About taking longer-term development and humanitarian scenarios into
account?
Information produced for emergency scenarios is different from information produced for
development activities. Up to now, FSNAU’s priority was to produce information for
humanitarian action, based on the food security assessments. Time, budget, security and
staff constraints hampered development of data appropriate for support of development
programmes. It is expected that this gap will be filled to some extent by the output of the
Livelihoods Research & Capacity Building section in the coming years.
IV. Synergies between FSNAU and SWALIM been created?
These synergies have been created but since SWALIM’s mandate is different from FSNAU’s
mandate the synergies are limited. SWALIM does not use “focal points” within national
Ministries and therefore the way SWALIM implements capacity building activities also differs
from FSNAU.
V. Contribution to improved knowledge on acute and chronic food insecurity and
malnutrition and their roots?
This knowledge is available but given its complexity it has not been presented in an optimal
way. Many factors play a role such as: i) FSNAU’s history – a merger of FSAU and the
Nutrition project in 2009 – which is also linked to the academic differences between the
concepts of food security (FAO and UNICEF); ii) the way “immediate” and “underlying”
causes are considered; iii) the fact that food security is concentrating on production,
livestock, migration, market developments and - according to a number of clients - less to
water, hygiene, health etc. factors.
VI. Improvement of capacity building: how, for whom and what?
FSNAU has a component of capacity building but both in Somaliland and Puntland no
capacity development officer is in charge. Capacity building is directly linked to the
sustainability of information collection and analysis. Focal points are used in the Ministries
both in Somaliland and in Puntland but despite their training FSNAU experiences a huge
staff turnover.
VII. Agreement on the issues that FSNAU is the institution that should continue to
determine the emergency or no-emergency situation in Somalia?
Given the fact that FSNAU is the major institution systematically collecting and analysing
national data on food and nutrition in Somalia, and the absence of any alternative this
question is not eminent. Adding to that the fact that short or medium term solutions for
Somalia are only desirable but not foreseeable, the only way to go forward is that FSNAU is
the institution which determines the emergency or no-emergency situation in Somalia.
Page 4
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
VIII. Relevance?
The relevance is in general recognized.
IX. Effectiveness?
The effectiveness is in general recognized.
X. Efficiency?
The efficiency can be improved taking into consideration that certain clients need more
“guidance” to understand the information, that information should be more tailored, that
information should be diminished in quantity etc.
XI. Impact?
The positive impact on “direct clients” is recognized, but the impact on “indirect clients” (the
needy population) has not, and cannot be verified with certainty.
XII. Sustainability?
Sustainability of a project such as FSNAU is difficult to achieve. Dependence on funding of a
number of donors means that once the donors decide to stop funding the FSNAU cannot
continue. In organised countries with a well-functioning administration, FSNAU tasks are
carried out by direct line ministries or semi-government agencies. This is not at all the case
in Somalia and only partially in the unrecognized countries of Somaliland and Puntland. By
means of its capacity building component, the project aims to build a structure that is able to
assume FSNAU’s responsibilities in due time.
XIII. Coherence?
The coherence of the project is not in question. Sections are well-organised and produce the
required information. However, the efficiency of the sections can be improved. The heads of
section spend a lot of time on editing reports and that time cannot be used for management
tasks. The question of methodology improvement is not directly related to the question of
coherence but it certainly is related to the transparency of the project such as is experienced
by a number of partners.
XIV. Added value specific to the European Commission?
The added value for the Commission is clear. FSNAU provides information to the DEU and
to ECHO. FSNAU is supported by the EU funded JRC.
4. Specific Recommendations.
I. Use of information?
Try to get a better insight into how information is used and by whom.
II. Appropriateness; and if enough tailored to local conditions and needs?
Make information simpler and more user friendly, in particular for clients who need
more training in how to read, understand and use the information.
Make the information more accessible for clients with different backgrounds.
III. About taking longer-term and humanitarian scenarios into account?
Try to find out in co-operation with clients which information for development is
needed, and if and how FSNAU can contribute to generating that information.
IV. Synergies between FSNAU and SWALIM been created?
Try to find out how capacity development assistance can be developed jointly.
Page 5
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
V. Contribution to improved knowledge on acute and chronic food insecurity and
malnutrition and their roots?
Work jointly with clusters on the question if and how indicators for “immediate” and
“underlying” causes can be identified and eventually can be integrated in the surveys
in order to identify in a better way the causes of acute and chronic food insecurity
and malnutrition.
VI. Improvement of capacity building how, for whom and what?
Try to find out with SWALIM how this activity can be integrated but only after finding
an alternative for the “focal points”.
VII. Agreement on the issues that FSNAU is the institution that should continue to
determine the emergency or no-emergency situation in Somalia?
Try to get full participation of clients – including those in Somalia - in the process of
validating the degree of emergency.
VIII. Relevance?
Continue the policy that clients also own the FSNAU as much as possible:
ownership.
IX. Effectiveness?
Take the concern of national partners regarding the stream of information and the
way they understand it seriously into account. Extra training with help of INGO’s is
necessary.
X. Efficiency?
Employ a full time editor for report writing etc. This will help to diminish the workload
of the heads of section and give them more time for management, training, coaching,
improvement of methodology etc.
XI. Impact?
Since impact on indirect clients (the food insecure population) is not measurable, try
to make a continuously updated inventory of impact of projects and programmes
using FSNAU information.
XII. Sustainability?
No direct recommendations are given but consider capacity building of national
institution as a contribution to sustainability.
XIII. Coherence?
Since the project is coherent enough, improvement of coherence must be sought in
the degree of participation of clients. One recommendation is to create a “technical
steering group” of partners that have legal influence on the used methodology.
Page 6
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Page 7
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
1. INTRODUCTION
This chapter is an introduction to the project, presenting the general context, the project, the
expected results, the methodological approach, the implementation and management
arrangements, the beneficiaries, the budget and the donors. It also explains the way the
evaluation has been carried out (methodology and programme) and the evaluation
questions. It aims to provide sufficient information to understand the conclusions and
recommendations worked out in the following chapters.
2.1. Description of the Project.
2.1.1. General Context.
Introduction.
The Somali people have now suffered the prolonged effects of civil war for more than twenty
years. Many attempts have been made by Somalis and the international community to
achieve peace and reconciliation in Somalia, but largely the country has remained
ungoverned since 1991. In the north of Somalia, two parts i.e. north-east (NE) Somalia or
Somaliland and north-west (NW) Somalia or Puntland have a self-declared independence
with a local administration – ministries etc. and a government – but Central and South (CS)
Somalia are still subject to rival clans and a rising insecurity due to the uprising activities of
the Al Shabaab movement. In 2004 the Transitional Federal Government (TFG) was
brokered by the Intergovernmental Authority on Development (IGAD) and the broader
international community, but it has failed to unite the country and has suffered from internal
divisions in the south. It is supported by the Ethiopian military which in itself generates
considerable resentment from within Somalia. Mogadishu remains very insecure and the
seat of government is still lodged to the south-west in Baidoa. Opposition to the government
has gained considerable momentum in recent years represented by both moderate and
fundamentalist elements. A political solution to the difficulties in Somalia seems as far off as
ever.
One particular exception to this situation is in the north-west of the country where there has
been relative stability for over ten years and a democratically elected government is well
established in Hargeisa. However, Somaliland has yet to gain formal international
recognition as an independent state, and its cause and stability has not been helped by the
conflict to the south. Neighbouring Somaliland is Puntland, in the north-east of the country,
which also declared itself a semi-autonomous state in 1998, but is now loosely associated
with the federal system of government in the south. There remains a long-standing and
fractious border dispute between Somaliland and Puntland which is unlikely to be resolved in
the current context.
Food, Nutrition and Livelihood Security in Somalia.
Somalia has a total land area of 637,540 square kilometres, a population estimated to range
between 7 and 8 million and the longest coast line in Africa. Whilst a semi-arid climate
prevails throughout the country, Somalia is characterised by a variety of livelihood zones
which are represented in the FSNAU map of Somalia, as shown below:
Page 8
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Page 9
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
The north and inland areas of Somalia are predominantly pastoralist areas combined with
fishing activities along the coastal belt. Somalia’s productive economy is predominantly
based on pastoralism and agro-pastoralism, so livestock has traditionally been Somalia’s
major export. However, in recent years livestock trade has been adversely affected by
restrictions on animal exports imposed by Saudi Arabia and other Gulf States. A
considerable amount of trade would otherwise take place within and through Somalia since it
provides critical sea access to Ethiopia in particular (along with Djibouti). The south, served
by the only two perennial rivers of Somalia, the Shabelle and the Juba, has two distinct rainy
seasons, the Gu and the Deyr, and is the main area for crop production. However, the
average annual cereal production has dropped significantly over the past 20 years since the
onset of conflict in the country and there has been an increasing dependence on commercial
imports and food aid. Only about 14% of the population is now engaged in cultivation based
agriculture. At rural household level, food security relies mainly on subsistence production
(livestock and crops) often supplemented by off-farm (sale of bush products) or non-farm
activities (labour and petty trade). Remittances from the diaspora (estimated at four to five
times the annual international aid budget to Somalia) are an important contributory factor to
food security, but more likely benefiting the higher wealth groups than the poor.
The south of the country has suffered in recent years from regular floods and drought which
have directly impacted on food security in the region. In early 2006, Somalia experienced the
worst drought in over a decade which affected pastures and food production.
By the end of 2006 the Deyr rains brought relief to drought affected areas, but then
excessive river flows caused widespread flooding in southern parts of the country not
experienced since 1997-98. Around 255,000 people in the Shabelle and Juba riverine areas
were displaced at that time. Now the country is again suffering from failed rains over the past
four seasons and facing a deepening drought and water crisis affecting all livelihood zones
including pastoral areas.
The years of conflict in Somalia has created a situation of protracted and complex
emergency which has eroded livelihoods and led to increased vulnerability to food insecurity.
In the last years there has been an escalation in the conflict between the Transitional
Federal Government (TFG) and insurgents of the opposition in the south of the country. This
has resulted in more than 870,000 people being displaced in the last year, many escaping
the insecurity in the capital Mogadishu, the worst over the past twenty years. It is also
creating an economic crisis throughout much of Somalia characterised by currency
devaluation, disrupted trade and market activities and hyperinflation of food and non-food
commodities. Within the first six months of 2008, the number of people requiring emergency
livelihood and humanitarian support increased 77% from 1.83m to 3.25m affecting an
unprecedented 43% of the entire population of the country. An increasing proportion of the
population affected are now urban-based. Food insecurity is strongly linked to poverty in
Somalia. Access to food and essential services (health, water, sanitation and basic
education) remains a major problem for Somalia’s most vulnerable groups: displaced
populations, ethnic minorities, subsistence farmers, urban poor and returnees. Throughout
central and southern Somalia, typical levels of acute malnutrition in children below the age of
five years, outside times of crisis, remain at over 15%, a level that would prompt major
emergency humanitarian interventions in other countries. In 2011, Al Shabaab imposed a
ban on all foreign aid activities in the area it controls (South-Central Somalia). In the north of
Somalia, malnutrition rates are somewhat lower although substantial pockets of high
vulnerability are seen in the urban centres, displaced peoples’ camps and areas
experiencing extreme environmental degradation. The underlying causes of the chronic
nutrition crisis are multi-sectoral, with fundamental factors such as lack of exclusive
breastfeeding, early introduction of complementary foods and very limited access to and use
of health services creating an underlying vulnerability. This coupled with limited access to
safe water and sanitation further predisposes the population to a poor environment for
Page 10
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
healthy growth and development which is exacerbated by chronic food insecurity and shocks
year after year.
Somalia has significantly the highest under five mortality rate and maternal mortality rate in
the region at 224 per 1,000 and 1,600 per 10,000 respectively which are key contributory
factors to Somalia being ranked 161 out of 163 in the Human Development Index (HDI).
2.1.2. The Project.
The Food Security Analysis Unit (FSAU) was first established in 1995 with funding from
USAID and the EU and implemented by WFP. Since early 2000, the FSAU project has been
managed by FAO, together with a sister Nutrition Surveillance Project. The two projects
merged in 2009 in the Food Security and Nutrition Analysis Unit (FSNAU).
The current project (Phase 6) runs from March 1st
2009 to February 28th
2013. This four-year
phase has a budget of $ 22.75 million; the five expected results are listed below.
The overall objective of the project is to ensure that at household level Somali food,
nutrition and livelihood security is strengthened, thereby ensuring greater resilience to
future “shocks” such as conflict, drought, flood, disease and inflation.
The purpose of the project is that a broad range of stakeholders has access to
appropriate food, nutrition and livelihood security information for improved
emergency response and development planning thereby ensuring that communities,
agencies and authorities in Somalia as well as the international aid community are
empowered to respond.
The project is based in Nairobi in the FAO Somalia office. It employs in its Nairobi office a
total number of 35 people and in the whole of Somalia 30 staff1
. The Nairobi office has a
Nutrition unit (7 staff members), a Food Security unit (4 staff members), a Livelihood &
Research unit (4 staff members) and two support units i.e. the Data processing management
unit (10 staff members), Publications sections (2 staff members) and the Operations and
Support unit (7 staff members).
In Somalia, Somaliland/North-West Somalia and Puntland/North-East Somalia the FSNAU
employs 16 field analysts on food security (1 is also liaison officer), 13 field analysts nutrition
(of which 5 are also focal points) and 1 capacity development officer.
Annex 3 gives an overview of the project’s management structure per section and per area
i.e. Nairobi, the CS (= Central South of Somalia), the NE (= North East of Somalia or
Puntland) and the NW (= the North West of Somalia or Somaliland)2
and indicates the
different titles employed.
The Nutrition unit is in charge of assessing the nutrition situation. In annex 12 the list of
nutrition monitoring indicators is given and in annex 10 the overview of the annual nutrition
surveys.
The Food Security unit is in charge of assessing the food security situation. Annex 11 gives
the list of the Food Security Monitoring Indicators and annex 9 the annual food security
surveys.
1
See in annex 13 the exact numbers of employees in Nairobi and in Somalia. Annex 14 shows the list of employees and their
titles.
2
The UN names Somaliland the North-West and Puntland the North-east of Somalia. The central and southern parts of
Somalia have the name CS Somalia.
Page 11
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
In general there are bi-annual seasonal food security and nutrition assessments – June/July
and November/December - and rapid assessments.
The section for Livelihood & Research is mainly in charge of research and studies – special
subjects – and gives support to the improvement of the assessment methodologies used by
the Food Security and Nutrition section.
The FSNAU produces an important number of bulletins, studies, updates etc. either through
their website www.fsnau.org or in hard copy form. See annexes 7 and 8.
The project has been delivering five results – see 2.1.3. - as planned. Given the continuation of
civil strife in Somalia, combined with severe food insecurity, malnutrition and famine across
several regions of the country, a further phase in support of the FSNAU (phase 7) is being
planned for another 4 years beyond February 2013.
2.1.3. Expected Results.
The project document mentions five expected results – see below – of which the core results
are: timely and relevant information and analysis; improved sector analysis and applied
research baseline livelihoods information and analysis, development of an integrated
database and capacity building of Somali institutions and partners with special focus on food
security, nutrition and livelihoods.
Result One: Timely and relevant food security, livelihood and nutrition information and
analysis provided on emergency situations
Result Two: Increased understanding of opportunities to reduce chronic food, livelihood and
nutrition insecurity through improved sector analysis and applied research on underlying
causes
Result Three: Baseline livelihoods information and analysis generated to inform the design
of early response and longer term interventions aimed at improving livelihood security
Result Four: Information is further organized, developed and incorporated into an integrated
database system and made accessible through managed information systems
Result Five: Technical capacity of Somali institutions and partners is strengthened in food
security, livelihoods and nutrition monitoring, assessment and analysis
N.B. These five results cover in general all activities carried out and all services provided by
FSNAU. It must be emphasized that both Early Warning and Emergency preparedness are
not included in the five expected results while often requested by clients and partners.
2.1.4. Methodological Approach of the Project.
In order to understand the meaning of food security and nutrition or better said “food and
nutrition security”, please read annex 15 which explains how to understand the different
concepts.
To assess the food security situation, FSNAU undertakes the following core activities:
1. Post-Gu and Post-Deyr Seasonal Food Security Assessments and Nutrition
Assessments
The seasonal food security assessments are conducted immediately after both rainy
seasons (the Gu main rainy season from April-June, and the Deyr short rains from October-
Page 12
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
December). They cover the entire country and identify areas and socio-economic groups
likely to face food security and nutrition problems during the coming period. The nutrition
surveys are conducted prior to the seasonal food security assessments and inform on the
nutrition and public health situation. These assessments support disaster preparedness and
provide food, nutrition and livelihood security analysis and trends information for long term
programming.
Outputs include: (i) bi-annual ICP food security projections; (ii) bi-annual nutrition situation
maps; and (iii) press releases, presentations, special briefs and bi-annual technical series on
seasonal assessment results.
2. Rapid Food Security and Nutrition Assessments
Rapid assessments are undertaken when a disaster has taken place such as a flood or
outbreak of disease, or when the security conditions do not allow regular surveys to be
carried out.
Outputs include: (i) press releases; (ii) presentations; and (iii) rapid assessment reports.
3. Food, livelihood and nutrition security monitoring and surveillance
Monthly or weekly community-level monitoring of the status of agriculture, pastoralism,
nutrition, health, water, markets, climate and civil security contributes to baseline analysis
and seasonal assessments as well as supporting the early warning function of the FSNAU.
Outputs include: (i) quarterly food security and nutrition briefs; (ii) monthly climate and
market data updates; (iii) bi-monthly nutrition updates; and (iv) monthly briefings to
coordination mechanisms.
4. Baseline Livelihoods Analysis
Baseline livelihoods analysis generates understanding of livelihood assets and strategies
and helps define longer term interventions aimed at improving livelihoods.
5. Applied thematic research
Applied research focuses on gaps in the understanding of underlying causes of food
insecurity and malnutrition; it informs longer-term policy and programming, and on support to
development/refinement of assessment methodology.
Livelihood based analysis
A household’s vulnerability to food insecurity may vary considerably according to its
livelihood. Therefore, the food security analyses are carried out by livelihood zone. To this
end, separate assessments – with separate sampling – are conducted in Internally Displace
People (IDP), urban and rural populations, and the rural area is divided into livelihood zones
characterized by the predominant livelihood, e.g. pastoral, agro-pastoral or
riverine/cultivation (see the map of the FSNAU livelihoods in Somalia on next page).
Seasonal nutrition assessments
If the security situation allows, the nutrition situation is assessed by means of representative
standard household surveys in the rural, urban and IDP populations. The methodology is
based on the SMART (Standardized Monitoring and Assessment of Transitional Relief)
guidelines3
which also provide the ENA (Emergency Nutrition Assessment) software module
for sampling and data processing. Households are selected by Region using two-stage
proportionate-to-size cluster sampling. Next, to enable livelihood based analysis, the sample
of a livelihood zone is created by clustering the selected and located communities in one
and the same livelihood zone.
Household questionnaires are used to collect data on household composition, morbidity,
mortality, food consumption, food security and access to water. Furthermore, the nutritional
status of under-five children and adult women is measured on the basis of internationally
accepted anthropometric indices, reference values and cut-off points. From these data, the
3
http://www.smartmethodology.org/
Page 13
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
prevalence of acute malnutrition is calculated, with global and severe acute malnutrition
(GAM and SAM) as key indicators. In case of time or security constraints, rapid nutrition
assessments are carried out instead of standard surveys. They consist mainly of
anthropometric data collection and a limited set of questions on morbidity and food security.
Due to security restrictions, updated nutrition and mortality data has not been collected in
December 2011 in Southern Somalia, apart from the IDP and urban population in
Mogadishu. However, indirect information on nutrition trends from health centres and feeding
programmes could be collected.
Seasonal food security assessments
Data on climate, civil insecurity, agriculture, livestock, markets, gender and nutrition of the
rural, urban and IDP populations are collected by means of a combination of standard
surveys, rapid assessments, monitoring and secondary data collection. Since the Gu 2011
seasonal assessment, representative standard household surveys are used for data
collection in the urban areas and IDP camps in areas with good access such as Somaliland
and Puntland. Random household selection will take place as described above. In the rural
areas, and in urban areas and IDP camps with restricted access due to insecurity, qualitative
rapid assessment methods such as focus group discussions and key informant interviews
are used. This is complemented by field observations by FSNAU field analysts (livestock
condition, crop), continuous monitoring by enumerators (market prices, terms of trade,
rainfall, etc.), remote sensing (rainfall, vegetation density), and secondary data (e.g. partner
reports)4
. Indicators obtained from these assessments are listed in annex 11.
Planning of seasonal assessments
An example of the total process of a seasonal food security assessment is presented in
annex 19. The assessment starts with a Technical Partner Planning Meeting in Nairobi to
discuss participation of partners, timeframe, tools, etc. Partners are UN agencies, INGOs
and LNGOs (with an office in Nairobi). Next, there are Regional Planning Workshops in
Somalia where local partners (NGOs and local authorities) are informed, their cooperation is
confirmed, and team composition and training are prepared. These workshops are led by the
regional FSNAU field analysts and the Nairobi-based food security analysts. Depending on a
Region’s size, the assessment is carried out by 2-4 teams working simultaneously, each
team consisting of a number of enumerators and one supervisor. The latter may be a
FSNAU field analyst or food security analyst as well as an experienced person from a UN
agency or NGO.
Data analysis
The objective of the analysis is to describe the current or imminent food security situation.
For this purpose, FSNAU uses the Integrated Food Security Phase Classification (IPC)
methodology, to the development of which it contributed largely over the past years5
. The
IPC is a set of protocols for standardized classification of the severity of food insecurity
situations. The IPC consolidates wide-ranging evidence on food insecure people to provide
answers to questions as: How severe? Where? How Many? Who? Why? Consensus
building by survey teams and experts on outcomes is one of the core characteristics of the
IPC process. The IPC is continuously improving its procedures. FSNAU used version 1.0
4
In regions of central and south Somalia where FSNAU has very limited access since the ban of Al Shabaab end 2011,
FSNAU field analysts undertake field observations of crop and livestock situation in all livelihoods but, due to insecurity,
commodity-level primary data is collected by enumerators and transmitted via teleconference to the FSNAU field analysts.
Despite the insecurity, FSNAU continued to receive routine monitoring data through markets and Somali Livelihood
Indicator Monitoring System (SLIMS) data points from all regions throughout the assessment periods.
5
The IPC is developed by an innovative multi-agency partnership of eight major UN agencies and international NGOs:
CARE International, the Joint Research Centre of the European Commission, FAO, FEWS NET, Oxfam GB, Save the
Children (UK&US), WFP that have joined forces to promote a „common language‟ to characterize the food security
situation, and more appropriate and effective policies and responses to food insecurity. More information on IPC is
available on www.ipcinfo.org.
Page 14
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
until the Post Gu 2011 assessment, and used the preliminary version 2.0 for its last
assessment of Post Deyr 2011-12.
FSNAU organises the IPC process along the following successive steps:
1. Regional Analysis Meetings Data entry starts while the assessments are still on-going.
Immediately after the field work, Regional Analysis Meetings are held simultaneously in
Hargeysa, Garowe and Mogadishu (situation Post-Deyr assessment 2011/12).
Supervised by FSNAU field analysts, the survey teams compile the multitude of evidence
from different sources in an ‘evidence based template’; they check the data for quality
and consistency with data from remote sensing, regular monthly field monitoring,
nutrition surveys, Somali Livelihood Indicator Monitoring System (SLIMS), main market
data, and previous trends; they also build technical consensus on the outcomes.
2. The Regional Analysis Meetings, usually in Hargeisa, are immediately followed by a
National Analysis Meeting, of two weeks with participation of the assessment teams of
all regions, and experts from INGOs, LNGOs, local government, UN agencies,
FEWSNET, JRC and FSNAU staff from Nairobi. The regional teams present their
outcomes by Region and by livelihood zone to the experts. They adjust the outcomes
where appropriate and assign a preliminary IPC class/phase. Next, this outcome is
presented to and challenged by the plenary bodies, followed by an ongoing process of
revision and challenge, resulting in a final IPC phase based on consensus. On average
half a day per Region is allocated to this process.
3. Vetting of the results in Nairobi. This is the final technical check of the analysis results of
the National Analysis Meeting by the FSNAU staff in collaboration with its primary
technical partners from LNGOs, INGOs and UN agencies. First, the nutrition situation
classification is vetted, followed by the vetting of the IPC results, which incorporate
nutrition and health indicators.
4. Estimating numbers of affected population by region, district and livelihood zone,
based on wealth categories and the deviation of key indicators (e.g. the Minimal
Expenditure Basket) from baseline values.
5. Press communications etc.
2.1.5. Implementation and Management Arrangements.
Institutional Framework and Coordination
The Project Management Advisory Committee (PMAC) meets twice per year to discuss
issues related to management, policy, strategy, new project documents, progress reports,
and evaluations. The membership includes the donors, UN agencies, the UN Resident
Coordinator to Somalia, the FAO Officer-in-Charge, the Chief Technical Adviser to the
FSNAU and FSNAU unit managers and the Lead Technical Unit to the Project Technical
Task Force.
The Project Task Force (PTF), comprising key FAO technical services/divisions such as the
Food Security and Agricultural Projects Analysis Unit (ESAF) of FAO and the nutrition and
consumer protection division at FAO (AGN), has an advisory function and coordinates
technical backstopping and technical missions to the FSNAU (sourced both from within and
outside FAO). The PTF also facilitates technical information exchange and supports the
FSNAU to identify institutional partners that could support the applied research programme.
The PTF meets twice per year in Rome.
Somalia Coordination FSNAU works through existing coordination systems to ensure that
the project is well integrated into established humanitarian and development planning and
information mechanisms for Somalia and engages with all key actors (including donors,
other UN agencies, NGOs and government authorities at a decentralised level). FSNAU
uses these meetings to share information and analysis, to verify findings, to tailor information
and analysis to user needs, and to advocate for appropriate, timely and integrated response.
Page 15
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Furthermore FSNAU contributes to the development of guidelines and strategies which
coordination working groups may be tasked to undertake.
The FSNAU provides between five to seven informal situation briefings every month through
various coordination mechanisms. The committees and working groups to which FSNAU
contributes most regularly include: Inter-agency Standing Committee (IASC) Steering
Committee; IASC Heads of Clusters Meeting; IASC Cluster Meetings (Nutrition Cluster
(FSNAU is co-chair); Health Cluster; Water, Sanitation and Hygiene Cluster; and Food
Security Cluster (FAO is co-chair)); Consolidated Appeal Process (CAP) workshops
facilitated by OCHA in Somalia and Nairobi; SSS (Somali Support Secretariat) Sectoral
Meetings; Zonal Livelihoods Sectoral Committee in the north-west and potential other zones
in Somalia.
Furthermore, FSNAU contributes to regional level forums such as the Food Security and
Nutrition Working Group (Horn and East Africa) of which FSNAU is a member; IASC Heads
of Clusters (global level); IPC Technical Steering Committee (global level); Nutrition Cluster
Assessment Working Group (global level); and the United Nations Standing Committee on
Nutrition (UNSCN) Nutrition in Emergencies Working Group (global level).
Other Technical Partnerships:
FEWSNET is directly supporting the costs of 24 enumerators for market and climate data,
provides lead support on climate and market analysis, and contributes to the overall analysis
on food security and livelihoods.
SWALIM and FSNAU are both key partners in developing the information base in Somalia
on food security, nutrition, livelihoods, water, land and the environment which are closely
interconnected. SWALIM is now managing river level monitoring and flood forecasting
(contributing to early warning), it generates data on water and land natural resources,
including soils, water, river catchment areas, agro-ecological zoning, (contributing to
baseline livelihoods asset and zoning analysis).
OCHA The relationship with OCHA ensures that food security, nutrition and livelihoods
analysis is effectively informing plans, strategies and the allocation of resources in support of
an effective and coordinated humanitarian response.
UNICEF FSNAU collaborates with UNICEF on the further enhancement of nutrition
information, nutrition assessments and methodology, monitoring of immunization coverage,
monitoring of water and sanitation indicators, malaria mapping and analysis, micro-nutrient
surveys, improved understanding of Knowledge Attitudes and Practices (KAP) to child care
and feeding, development of International Education Center (IEC) materials on child care
and feeding and the Nutrition Information Project for the Horn of Africa (NIPHORN) which
supports local health administrations to expand the capacity for nutrition surveillance in
Somalia.
Additionally FSNAU has close working relationships with the WFP Vulnerability and
Mapping (VAM) unit and CARE which rely significantly on FSNAU data for food aid
projections, and the World Bank on economic indicators including recent interest in urban
market information. The Nutrition team liaise closely with the Institute of Child Health and
Centers for Disease Control (CDC) on technical issues relating to sampling and
assessment methodology.
The Joint Research Centre (JRC) of the European Commission (EC) trains FSNAU in the
application of remote sensing for monitoring rainfall and agricultural vegetation (including
pastures), and yield forecasting.
FSNAU has a broad network of staff within Somalia in spite of the current security crisis. The
field team of food security and nutrition analysts and enumerators represent the backbone of
the unit and integral part of the technical team. The field team’s configuration is as follows:
• 105 part-time food security enumerators (of which 12 enumerators are funded by FEWS-
NET) responsible for collecting primary data on markets, rainfall, key SLIMS indicators and
other supplementary information;
Page 16
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
• 20 part-time nutrition enumerators responsible for collecting health facility based nutrition
data and involvement in assessments on an ad hoc basis;
• 16 food security field analysts (including up to four liaison officers) who will review,
collate and integrate the data from enumerators, undertake sectoral analysis in their area
and facilitate baseline analysis and seasonal assessments.
• 14 nutrition field analysts (including five focal points) responsible for the monitoring and
reporting of the nutrition situation in their area, conducting rapid assessments and training of
partners on nutrition surveillance, micronutrient deficiency, food processing and
preservation.
N.B. Capacity development officers are both planned in Hargeisa and in Garowe. A new
one has to be appointed in Hargeisa. In Garowe no one has been appointed.
2.1.6. Beneficiaries.
The project’s general objective indicates to ensure the Somali food, nutrition and
livelihood security at the household level. Greater resilience to future “shocks” such as
conflict, drought, flood, disease and inflation must be guaranteed. Also a broad range of
stakeholders must have access to appropriate food, nutrition and livelihood security
information for improved emergency response and development planning thereby ensuring
that communities, agencies and authorities in Somalia as well as the international aid
community are empowered to respond.
The project document mentions that the ultimate beneficiaries to the FSNAU project are the
Somali populations who are vulnerable to food and nutrition insecurity. They will benefit from
the informed and strategic response of humanitarian and development actors in Somalia.
The primary beneficiaries of FSNAU information are the Nairobi-based international
community, notably donors, UN agencies, international and Somali NGOs.
Particular beneficiaries in the different parts of Somalia are:
• Emerging national government structures concerned with nutrition and food security;
• Somali Academic Institutions who focus on food security and nutrition education;
• Commissions or agencies charged with contingency planning or emergency
preparedness (NERAD, HADMA);
• Non-Government Organisations and Civil Society Organisations requiring local
information for strategic planning and monitoring purposes.
2.1.7. Budget and Donors.
The total budget for phase 6 (2009 – 2013 or 4 years) amounts a total of 22,750,000 US$.
Major donors are the SIDA, the EU, the USAID, the DFID and minor donors are SDC, Italian
Cooperation, UNICEF, UNHCR, OCHA-CHF6
.
Result 1 – the timely information and analysis - covers 50% of the budget.
The breakdown of the budget is according to the budget proposed and approved of in the
project document:
Salaries professionals, technical consulting staff etc. 49%
6
6,000,000 US$ by SIDA, 3,900,000 US$ by EU, 5,000,000 US$ by USAID, 3,000,000 by DFID and
1,500,000 US$ by OCHA-CHF, 900,000 US$ by UNICEF, 595,000 US$ by UNHCR, 300,000 US$ by
the SDC and 300,000 US$ by the Italian Cooperation.
Page 17
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Travel 23%
Contracts 7%
Operational expenditure 7%
Training, capacity building etc. 5%
Rest: (non)-expendable procurements, technical support etc. 9%
2.2. Description of the Evaluation.
The methodology used during this evaluation followed traditional procedures. The basis for
the evaluation was the Terms of Reference in which the general and specific objectives of
the evaluation have been given.
The ToR objective lists four key evaluation questions to be answered, and specifies four
specific aspects – related to the DAC evaluation criteria - of the project to be assessed (see
2.3.).
The required output of the evaluation is a draft evaluation report and a final evaluation
report. An inception report is not requested.
Although not requested, the evaluation team has written a concise inception report of five
pages in which a number of questions have been formulated according to the ToR,
including a specifically requested response by the management team of the FSNAU and an
additional minor question requested by the DEU Somalia unit evaluation task manager.
The first four weeks were mainly used for the inception and for the interviews with all donors,
the UN agencies and the main NGOs which are partners in the field and also involved in
data collection. The second week the team travelled to Somaliland and Puntland where all
Ministries and partners – UN and NGOs – were interviewed including all focal points. The
fifth week was used for wrapping up and included a debriefing organized for all stakeholders
and clients, whether they had been interviewed or not.
The interviews were carried out in a semi-structured way using a list of 16 questions mainly
used as a discussion guide and presented below.
1. How do and did you use FSNAU information?
2. What specific information do and did you use?
3. What information do you collect yourself?
4. Does FSNAU cover all fields regarding food and nutrition?
5. If not what should it cover in addition?
6. What information do you miss that FSNAU does not produce?
7. Specific information about political and technical position of FSNAU?
8. Information about the FSNAU methodology?
9. Information about the FSNAU validation quality of information?
10. How can FSNAU improve their performance?
11. Information about the FSNAU structure?
12. Expectations of the FSNAU steering committee?
13. How to improve capacity building, for whom and what?
14. What do you think of the sustainability of FSNAU?
15. Specific recommendations for this evaluation and related to the ToR?
16. Specific wishes of donors and partners?
The answers to the questions were worked out in a table presenting the observations and
concerns expressed by the interviewees. Annex 16 presents an overview of answers to
these questions.
Page 18
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
2.3. Evaluation Questions.
A total number of 23 Evaluation Questions have been formulated and answered in chapter 3.
The questions are classified according to: the Terms of Reference, FSNAU management,
DEU Somalia unit and DAC Criteria. Annex 17 gives the detailed questions and sub-
questions to be answered.
Questions according to the TOR, the FSNAU management and the DEU.
1. To what extent has food security and nutrition information and analysis produced by
FSNAU been used (by decision makers, researchers, implementing agencies and civil
society Somali and non-Somali) and for what purposes?
2. To what extent are FSNAU activities appropriate and tailored to local conditions and
needs? Are they carried out taking longer-term development and humanitarian scenarios
into account?
3. Has FSNAU and FAO created synergies between information system projects of FSNAU
and SWALIM?
4. To what extent has FSNAU contributed to improved knowledge on acute and chronic
food insecurity and malnutrition and their root causes in Somalia.
5. How can the capacity building component be improved, for whom and what?
6. What has been done well, what not, and how can that be improved?
7. What do partners want in the next phase?
8. How has the information been used by the partners?
9. Do we agree that FSNAU is the institution that should continue to determine the
emergency/no-emergency situation of Somalia (a sort of switch on, switch off)7
.
Questions according to DAC criteria.
Relevance of the project
10. To what extent are the objectives of the project still valid?
11. Are the activities and outputs of the programme consistent with the overall goal and the
attainment of its objectives?
12. Are the activities and outputs of the project consistent with the intended impacts and
effects?
Effectiveness of the project
13. To what extent were the objectives achieved / are likely to be achieved?
14. What were the major factors influencing the achievement or non-achievement of the
objectives?
Efficiency of the project
15. Were objectives achieved on time?
16. Was the programme or project implemented in the most efficient way compared to
alternatives?
Impact of the project
17. What has happened as a result of the project?
18. What real difference has the activity made to the beneficiaries?
19. How many people have been affected?
7
In the last 4 years or more the absent of a Somali state in the south has generated an uncomfortable feeling that FSNAU is
replacing an important function of a state to call for assistance or not call for assistance (see Eritrea). How do stakeholders
feel on the important political power that we have delegated to FSNAU? Can they manage it? Is the system capable?
Page 19
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Sustainability of the project
20. To what extent did the benefits of the project continue after donor funding ceased?
21. What were the major factors which influenced the achievement or non-achievement of
sustainability of the project?
Coherence of the project
22. What is the logical, orderly and consistent relation of parts of the project?
Added value of the project specific to the Commission
23. Is there added value and if yes, what added value of the project?
Page 20
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Page 21
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
2. FINDINGS
In this chapter evaluation questions and evaluation criteria are analysed and answers are
given. The origin of the evaluation questions and evaluation criteria are from the ToR
(questions 1 – 5), the FSNAU management team (questions 6-8), the DEU (question 9) and
the criteria questions according to the DAC8
as formulated in the ToR (questions 10 – 23).
Evaluation Questions and Criteria.
According to the Terms of reference, the FSNAU management and the DEU:
1. To what extent has food security and nutrition information and analysis produced
by FSNAU been used (by decision makers, researchers, implementing agencies
and civil society Somali and no-Somali) and for what purposes?
FSNAU produces and analyses general Food Security and Nutrition information on the
whole territory of Somalia – Somaliland, Puntland and CS Somalia – in general per livelihood
zone, for the category of internally displaced persons (IDP), for the urban and for the rural
population on a bi-annual basis. This information is published via the FSNAU website
(www.fsnau.org) and a series of technical reports, monthly and quarterly briefs, monthly
update bulletins for Nutrition, for Markets, for Climate, baseline reports etc. (see annex 7:
FSNAU reports and publications and annex 8: Technical Reports Generated During the
Reporting Period phase 6). The users or clients are mainly the donors, the specialized or
research agencies, the programme implementing agencies but also the (non)-governmental
agencies in Somalia and the civil society in either Somalia or somewhere else. In Somaliland
and Puntland the users are all local Ministries, the NERAD, the HADMA and the national
NGOs. The information is mainly used by the implementing agencies such as international
and Somali NGOs for preparing new project proposals, by donors to justify their support, by
UN agencies to be informed but also to formulate their support and fields of special attention,
and by the UN and NGO community to launch the yearly consolidated appeals (CAP) as well
as allocate funds of the Common Humanitarian Fund (CHF), etc. The information is
according to the clients “enough” and “sufficient”; some would like to see more (frequent
updates), and to others it is “too much” and not sufficiently structured in a way to understand
or to find the information needed. For some clients the great quantity of information makes it
that “You cannot see the wood for the trees”. Some clients prefer that food security and/or
nutrition data collected by themselves in ‘pockets’ of the livelihood zones or regions/districts,
be incorporated in the FSNAU reports, or at least be considered in the analysis of the area.
FSNAU argues that information of clients can only be integrated if their methodological
validity has been proven.
In general the information provided is qualified as very useful to useful.
2. To what extent are FSNAU activities appropriate and tailored to local conditions
and needs? Are they carried out taking longer-term development and
humanitarian scenarios into account?
Appropriateness of activities
Because of the long absence of an effective government in Somalia, it was inevitable that
the UN assumed responsibility for the coordination of the humanitarian programme,
including FSNAU’s core task of providing the information required for decision making on
humanitarian intervention programmes. Due to the prevailing civil insecurity in Somalia, the
decision to operate FSNAU from Nairobi is appropriate. However, if the situation in
8
Definitions of DAC criteria are given in Annex 10.
Page 22
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Somaliland and Puntland remains stable, decentralisation of (part of the) FSNAU activities to
these areas should be considered, as data collection and analysis is likely to benefit from
closer contact with the partners (local government and (I)NGOs). It will also enhance
opportunities for capacity building.
FSNAU uses a wide range of data collection tools that are utilised according to the prevailing
conditions. For example, when time or security conditions do not allow standard household
surveys to be used, it employs rapid assessments to collect data. Other examples are the
use of digital forms to speed up data processing and reporting, and the use of
telephones/teleconferencing to transfer data from the field since the ban of Al Shabaab in
2011 made normal operations of FSNAU field staff almost impossible. Some partners have
doubts about the reliability of the data transferred by telephone. However, according to
FSNAU there is not much reason for concern because the data are cross-validated with data
from other sources and obtained in the same period.
Some UN agencies and NGOs feel that the data in the FSNAU assessment reports do not
provide enough geographical detail to be used as a basis for designing intervention
programmes. This is true, but it should be acknowledged that the assessments have not
been designed for that purpose. They have been designed to provide a general picture of
the severity and magnitude of the food security situation by Region, District or Livelihood
zone, and the sample size does not allow analysis at a local level. Implementing agencies
themselves have to carry out local in depth investigations, if needed with technical support of
partner (I)NGOs. It is conceivable, however, that in future FSNAU, in collaboration with these
(I)NGOs, trains the small local NGOs in assessment methodology. FSNAU is already
training LNGOs to conduct quality nutrition surveys.
In line with the IPC data analysis protocol, FSNAU has made a great effort to involve all of its
partners in the data gathering and analysis processes from the initial to the final stage.
Although this is generally appreciated by the partners, some concerns were raised by Somali
partners. Local government officers said to regret the fact that the final IPC classification
step was made in their absence at the vetting in Nairobi. Although they are encouraged by
FSNAU to participate in the vetting, and recognize that possible amendments in
classification outcomes were relatively rare and comparatively small, it gave them a sense of
loss of ownership. To some extent this also seemed to be the case for some Somali NGOs
for whom, just like the government officers, there are practical constraints (time, budget for
travel and accommodation) to participation in the vetting or even in the National Analysis
Meeting in Hargeisa.
The outcomes of the food security assessments are frequently and timely published in a
range of publications, released as hard copies and as digital files accessible from the
FSNAU website. These publications are very much appreciated by most of FSNAU’s clients.
However, some of them indicated to have difficulty in finding the right type of information
and/or in understanding it. This applies particularly to the NGOs and governmental
institutions in north Somalia.
By and large, it is concluded that the activities and methods employed by FSNAU are
appropriate and tailored to the prevailing conditions and the needs of its clients.
Consideration of longer-term development and humanitarian scenarios
FSNAU’s activities are focused on food security situation analysis with a view to supporting
the design and effectiveness of emergency humanitarian interventions, rather than longer-
term development oriented programmes. Therefore, it says it focuses on the immediate
causes rather than the underlying causes of food, nutrition and livelihood insecurity. This
also explains why FSNAU rightly does not see it as its primary responsibility to carry out
certain monitoring activities, e.g. monitoring of water sources, as requested by some
partners. The reason for these requests is understandable: FSNAU is the only agency
having a country-wide, long-lasting and fairly robust data collection and analysis system.
However, besides the above fundamental argument against taking up additional data
gathering, the current heavy work load of FSNAU does not allow it to do more work. It would
be interesting to see if data collection in support of development planning could be taken up
Page 23
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
by other organisations, starting for example in north Somalia and eventually with some
guidance from FSNAU.
All this does not at all imply that the information generated by FSNAU is not suitable for
recovery, rehabilitation or development planning. Although focused on the immediate causes
of food insecurity, the assessment reports of FSNAU inevitably also provide insight into the
deeper and chronic causes of food insecurity. Moreover, this type of information is also
available from the baseline livelihood studies and special research reports of the FSNAU
Research and Capacity Building unit.
3. Has FSNAU and FAO created synergies between information system projects of
FSNAU and SWALIM?
In the past years, SWALIM and FSNAU have cooperated in the following fields:
 SWALIM and FSNAU have improved their skills in drought (rainfall) and vegetation
monitoring by remote sensing, with technical support of JRC. Jointly with FEWS NET
they contribute to the climate sections in publications. It is currently being investigated if
remote sensing can also be used for crop forecasting.
 FSNAU feeds the flood monitoring system of SWALIM by providing field data on flood
impact.
 A study initiated by FSNAU on the iodine content of drinking water has been shared with
SWALIM
 FSNAU financed a study by SWALIM on arable land.
 A platform has been created which enables SWALIM and FSNAU to have access to
each other’s databases.
Although collaboration in some fields has been achieved, both projects acknowledge that it
will be difficult to achieve more, given the big difference in nature and mandate of the
projects. SWALIM sees a potential role for FSNAU in monitoring water sources but, as
explained above, it is difficult for FSNAU to assume this role. According to FSNAU,
collaboration is more likely to develop through its Research and Capacity Building section.
This would be in support of a suggestion by SWALIM that there may be opportunities for
cooperation in capacity/institution building in north Somalia where SWALIM is already
supporting three ministries in Somaliland. The current project phases of FSNAU and
SWALIM expire almost simultaneously around the end of this year. As suggested by
SWALIM, this offers an opportunity to explore opportunities for collaboration in the next
phase, and align the project proposals accordingly.
In conclusion: there has been more collaboration between FSNAU and SWALIM. There is
scope for further collaboration in the use of satellite imaging and possibly in the field of
capacity/institution building.
4. To what extent has FSNAU contributed to improve knowledge on acute and
chronic food insecurity and malnutrition and their root causes in Somalia?
Acute and chronic malnutrition refers in general to a situation in which under-five children are
“wasted” according to the relation between their weight and height (acute) and stunted once
the growth or height of the child is stunting in relation to its age. Acute or chronic food
insecurity has to do with the acute or chronic availability and access to food of good quality
and nutritional value etc.
FAO and UNICEF use different definitions for the concept of food security (see annex 15)
but in general UNICEF is the more nutrition and health related component.
However, FSNAU uses the more FAO oriented approach and concept but it must be
recognized that the nutrition component has been taken considerably into consideration. It
Page 24
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
must be recognized that certain WASH but also health related indicators can be taken into
consideration in a more efficient way. We must recognize that FSNAU must take into
account a difference between the “immediate” and “underlying “causes of food insecurity and
malnutrition. It is their task to assess the “immediate” causes but not the “underlying”
causes. The WASH and also the nutrition cluster’s wish to include a number of indicators is
justifiable but FSNAU jointly with the two clusters have to sort it out in a positive way how a
distinction can be made between the indicators taking into consideration the difference
between “immediate” and “underlying” into consideration. It is important that FSNAU and the
WASH and Nutrition clusters agree on this issue of the final use of indicators and the weight
of each indicator in the process of validation.
The question of “root causes” of food insecurity and malnutrition in Somalia is both related to
the immediate and underlying causes. It is both food and health related but in what degree is
not clear because in “emergency” situations and “normal” situations other factors also must
be taken into consideration.
5. How can the capacity building component be improved, for whom and what?
Building capacity of the Somali government
In a ‘normal country’ the government takes care of the work that is now being done by
FSNAU. Indeed, in Africa there are examples of countries that have government institutions
conducting food security monitoring and analysis. However, due to the prolonged quasi-
absence of a functioning government as a result of civil strife, these institutions do not exist
in Somalia. FSNAU has shown to be committed to filling this institutional gap by ‘….. building
the technical capacity of Somali institutions where suitable opportunities arise. This builds
upon the system of focal points identified in government ministries and FSNAU training
initiatives already undertaken in phase V’9
. The Somali institutions benefiting now from this
capacity building activity through ‘focal points’ are a number of line ministries and
government agencies in the regions of Somalia which have been relatively stable and quiet
for a longer period of time, i.e. Somaliland and Puntland. The National Environmental
Research and Disaster Preparedness Commission (NERAD) in Somaliland and the
Humanitarian Affairs and Disaster Management Agency (HADMA) in Puntland are among
the supported agencies. The training of the focal points consists of a computerised training
package for self-learning, and training-on-the-job by their participation in the seasonal
assessments. According to FSNAU and the ministries, the focal points are employees of the
benefiting institutions and they have been carefully selected on the basis of their level of
education, experience and motivation.
At meetings of the evaluators with ministries, focal points and UN agencies in Somaliland
and Puntland, the following emerged:
 The governments expressed their desire to accept more responsibility for food security
monitoring and analysis, and the need for adequate technical capacity to be build.
 The UN agencies feel that it is appropriate now to move more of FSNAU’s work which is
currently being done in Nairobi, to north Somalia and build technical capacity at the
government level.
 There is no clear, unanimous opinion about the modalities of the required food security
information structure. Should it be: attached to, or part of a line ministry?; a strengthened
NERAD/HADMA?; a new, autonomous institution focusing on food security?; or should it
be part of an institution having a wider scope, e.g. a national statistics office from which
clients can draw the information they need? Whatever the structure, most interviewees
felt that it must be independent/objective and have a strong data management capacity.
9
Quote from the project document.
Page 25
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
 The Somaliland and Puntland governments do not have the financial means for the
structure and would need to be supported for a long period of time.
 Building appropriate technical capacity requires a long-term programme of institution
building.
 The ministries and focal points were generally happy with the current capacity building
activity. However, the focal points indicated the need for face-to-face contact with
trainers along with the computerised module for self-training.
Although it is clear that FSNAU will still have to do the food security assessments in the
foreseeable future, there are opportunities to start capacity building in north Somalia. The
focal points approach may be a good way for FSNAU to establish and maintain contacts with
the ministries and agencies. Therefore it merits to be maintained for the time being.
However, it probably is not an effective and efficient step on the way to institution building
because the position of the focal points within the line ministries is too isolated. Moreover,
trained people tend to disappear once they have completed their training. It was even noted
that some of the focal points were not employed by the ministry before being appointed as a
focal point (they worked at the ministry as a volunteer).
A comprehensive institution building programme requires a wide variety of inputs which
FSNAU cannot deliver alone. Therefore, it requires the involvement of partners as well.
A general point of concern in relation to wider government involvement is the risk of the
analysis becoming politicised. To minimise this risk, it is recommended to keep the
government as well as other partners involved in the conduct of the whole assessment -
from the start till the end - and to emphasize the need for rigorous, evidence based analysis.
Building capacity of civil society organisations
There is a need to train Somali NGOs in the ability to conduct nutrition surveys and food
security as well as needs assessments. FSNAU is already involved in such training and this
needs to be followed up. However, this training should not and cannot be done by FSNAU
alone. Most of the smaller Somali NGOs have partnerships with INGOs or bigger Somali
NGOs which, to a greater or lesser extent, (should) already dispose of technical capacity in
this field, and therefore should also play an important role in the training programme.
6. What has been done well, what not, and how can that be improved?
The concerns and observations (comments) made by a number of clients are not due to a
perceived lack of performance of FSNAU, but other reasons were mentioned. The
information provided by FSNAU has generally been evaluated as useful for clients.
Concerns exist about the reliability of data collected in CS Somalia and transferred by mobile
phone. How sure is FSNAU that these data have been collected in a reliable way? Another
concern is that results are too general, for instance published for the whole livelihood area,
and data from specific pockets within these livelihoods produced by individual agencies were
not taken into consideration. A number of clients have expressed the wish to include these
data in order to get a more complete picture of the food and nutrition situation in that specific
livelihood. According to them, FSNAU does not include these data merely because the
validity cannot be controlled. Another concern is the way some information has been
collected, among others by FSNAU, and about its validity and objectivity.
Main concerns exist about two major themes i.e.
1. The great quantity of information produced.
2. The skills required to read and to understand it.
These concerns came mainly from clients working in Somalia in the field but also from a
number of Ministries in Somaliland and Puntland. The great quantity of information produced
Page 26
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
by FSNAU is a reality and FSNAU should find out how this information can be provided more
systematically and how its quantity can be reduced. This makes the need for more tailored
information an option: who needs what information? What is information of primary need and
what of secondary or tertiary need and for whom?
If information is produced in less quantity and more tailored – this needs a detailed needs
assessment – technical staff need less time to produce the big stream of information and will
make time free for other essential duties such as training and coaching of employees of all
clients included FSNAU staff. This is not only a concern of the evaluation team but also of a
number of clients. The second concern is directly and indirectly related to the first concern. A
number of clients expressed the need of more capacity building in order to understand the
FSNAU data better. They were mostly clients based in the field and in the Ministries in
Somaliland and Puntland who expressed this need. FSNAU has to find out what they do not
understand and how produced information can either be provided in a more tailored way or
in a more understandable way. One option is to produce a general manual which explains in
detail how to read the FSNAU information10
. This manual could possibly give an introduction
to the way FSNAU is organised, what FSNAU is doing, explain the used methodologies, and
the way data can be interpreted and used. FSNAU works for a range of clients with different
capacities and needs.
7. What do partners want in the next phase?
Partners or clients are not requesting big changes or adaptations. A number of changes
have been expressed already in the answers to previous evaluation questions but we will
summarize them once again.
 Integration of WASH and basic health indicators directly related to food security and
nutrition in the questionnaires;
 Integration of detailed information from specific pockets in a livelihood zone given by
clients and after verifying the validity of these data by FSNAU;
 Less and more tailored information and in a more structured way;
 Capacity building of a number of clients in order to make them able to read and
understand the FSNAU information and to conduct quality assessments/surveys;
 Evaluate the effectiveness of Focal Points within the Ministries and eventual alternatives;
 Review of FSNAU concept of Food Security and Nutrition etc. and more change towards
an integral concept of Food and Nutrition Security;
 More participation in the improvement of methodologies;
8. How has the information been used by the partners?
Most partners use the FSNAU data as a basis for formulating proposals for project funding
and for planning interventions. According to some of FSNAU’s partners, donors do not even
accept proposals for funding if they are not based on FSNAU data. Organisations that
conduct surveys themselves, e.g. the International Committee of the Red Cross ICRC, use
the FSNAU data for triangulation. HADMA (Puntland) said to conduct (inter-agency) needs
assessments in areas classified by FSNAU as severely food insecure.
10
The evaluators experienced the need of structuring and tailoring the produced information when assessing the ways the
various assessments are carried out and in particular the methodology used. It took long time to find out what the different
methodologies were and how the assessments are organized. A simple but detailed manual explaining in detail what is done
by whom and how etc. makes all procedures more transparent.
Page 27
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
9. Do we agree that FSNAU is the institution that should continue to determine the
emergency/no emergency situation of Somalia?
Taking the official objective, the purpose and the five expected results seriously into account,
it is not FSNAU’s mandate to determine the emergency or no-emergency situation of
Somalia. However, the fact is that no other structure exists to do this. The information
FSNAU provides and the use of the IPC classification brings FSNAU in a position to be the
only structure to be able to declare emergency/no-emergency. The overall objective
mentions to ensure that the “Somali food, nutrition and livelihood security at the household
level is strengthened thereby ensuring greater resilience to future shocks” and the purpose is
“that a broad range of stakeholders have access to appropriate food, nutrition and livelihood
security information for improved emergency response and development planning”. The five
expected results will mention mainly “timely and relevant information for increased
understanding of opportunities”, ”through good sector analysis and research to reduce
chronic food, livelihood and nutrition insecurity”, “information and analysis to inform the
design of early response and longer term development”, the organisation of a professional
data system” and “capacity strengthening or building of Somali institutions and partners”.
Therefore the answer to this question is easy: Yes. FSNAU is the only capable institution in
the absence of a qualified government structure. Therefore FSNAU must continue to assess
and classify the food security situation, involving all of its partners as much as possible.
DAC criteria:
Relevance of the project
10. To what extent are the objectives of the project still valid?
Evaluation question 9 explains the validity of the objective and purpose of the project in such
a way that it clarifies the duties of FSNAU and also its mandate. It is up to the clients -
donors, Somalia institution, UN agencies, NGO’s etc. – to determine what institution should
be able to declare the state of emergency or not. From a technical point of view FSNAU is
certainly competent to do this but from a more political point of view this is questionable
because it gives FSNAU a very heavy responsibility.
In general, the overall objective and the project purpose are still valid because the situation
in Somalia has not changed or - better said - improved over the last years.
11. Are the activities and outputs of the programme consistent with the over-all goal
and the attainment of its objectives?
The main activities concern data collection, analysis and publication. This is also what has
been mentioned as the general objective and the main purpose. In the five expected results
it has been worked out how the collected information and analysis must be used and for
what purposes, including the capacity building and the general support FSNAU is giving to
the clients. There is no inconsistency between the activities and outputs on the one hand
and the overall goal and attainment of its objectives on the other. The only concern is how to
make information more accessible and more simple, and also to increase the transparency
of the methodological approach of the project.
Generally this consistency exists. The activities are all geared to generating quality
information (output) relevant to emergency response planning by programme implementing
partners and donors. However, some partners (particularly Somali NGOs and government)
would like the reports to be written in a more comprehensible language.
Page 28
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
12. Are the activities and outputs of the project consistent with the intended impacts
and effects?
Yes. By and large, the clients of FSNAU are satisfied with the information provided by
FSNAU, and they actually use that information as a basis for designing emergency
interventions to strengthen the food security situation of Somali households. The extent to
which the interventions have actually led to improved household food security cannot be
determined with certainty, but it is very likely that they have been beneficial. Obviously, the
actual food security situation of households does not depend on FSNAU food security
analysis alone, but also on factors beyond control of FSNAU.
Effectiveness of the project
13. To what extent were the objectives achieved or are likely to be achieved?
The general objective of the project is to ensure the food, livelihood and nutrition security in
Somalia etc., but it has already been explained in evaluation question 12 that achievement
of the overall objective is difficult to measure. It is also explained why. This question can be
answered more easily for the purpose of the project i.e. “access by stakeholders to data for
improved emergency response and development planning”. This is fully achieved but some
recommendations have been given on how to improve but also to complete the information.
It concerns mainly the wishes of the WASH and Nutrition clusters to include extra indicators
related to the “immediate causes of malnutrition”. The issue of the broad range of
publications has already been mentioned but also the need to structure the information in a
better and more logical way. In order to prevent clients from saying “You cannot see the
wood for the trees”, the FSNAU must analyse the following question very seriously: “How
can we simplify the information in a more structured and understandable way?”. Maybe the
need for a professional editor must be taken seriously into consideration in order to release
the heads of sections from having to edit the great number of reports etc. they produce, and
to help them to structure and simplify the number of publications.
14. What were the major factors influencing the achievement or non-achievement of
the objectives?
The factors influencing the achievement of the objectives and purposes are:
 Sufficient means and budget;
 Well qualified technical staff;
 Good cooperation with clients: UN-agencies, INGO’s, Somali government staff and
donors;
 Good technical support from the Rome based Technical Task Force TTF and other UN
agencies and NGOs;
 Good management;
 Enough confidence from donors’ side;
 Involvement of all clients in the validation procedures of data collection and analysis;
 Factors influencing non-achievement: extreme civil insecurity, leading to extremely
restricted access to the field for data collection.
Page 29
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
Efficiency of the project
15. Were objectives achieved on time?
There is no clear time path or planning for achievement of objectives and purposes. The
project exists since 1995 and the nutrition unit was merged into the FSAU in 2009. There
have not been serious concerns or complains about non- achievement of the objectives,
purposes or expected results beside that some information comes rather late.
The project has no predetermined end point. Its services are required as long as food
insecurity lasts, and this depends to a great extent on the civil insecurity that prevails in the
country. FSNAU regularly updates its data and information to allow the humanitarian
community to adjust its interventions to the prevailing conditions.
Except from being sometimes late – this could not be verified by the evaluators – the data
and information are generally updated in time.
16. Was the programme or project implemented in the most efficient way compared to
alternatives?
Alternatives are not known and other structures implementing such a project for Somalia do
not exist. It has been mentioned already that more efficiency can be achieved if more
structured and more tailor-made information will be provided. There is a need for a simple
manual introducing outsiders in a more efficient way to the project, its organisation, its
methodology, the frequency of all assessments etc. etc. The need for an editor has also
been mentioned to release heads of sections but also to help FSNAU to systematise their
different types of information.
Impact of the project
17. What has happened as a result of the project?
As a result of the project, donors were able to allocate their funds in a rational way, and
programme implementing organisations were able to substantiate funding proposals and
identify relevant and appropriate interventions.
Out of the five expected results mentioned in chapter 2.1.3., the first four have been
achieved “grosso modo”. The last result i.e. the capacity building should be given more
consideration mainly for national government partners and for NGOs mainly working in the
field. SWALIM applies a different way of capacity building, not based on Focal Points.
FSNAU has to assess this and alternative ways jointly with SWALIM and other potential
partners.
18. What real difference has the activity made to the beneficiaries?
It has already been explained that the impact on the indirect beneficiaries – IDP’s, vulnerable
families, drought victims, malnourished children – can hardly be measured.
For the direct beneficiaries (donors, agencies etc.) the impact has been quite positive.
19. How many people have been affected?
What is the meaning of the term ‘affected people’ and affected by what? Somalia suffers
from a multitude of problems which affect people: civil war, insecurity, drought, floods, lack of
facilities such as health and education, no governance etc. The total population of the
Somalia area is not known, the numbers of IDP’s can only be estimated, the number of
Page 30
Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012
Apr
refugees living in neighbouring countries can be calculated, the number of Somalis in the
diaspora can also be calculated. The number of people affected differs from period to period
and is estimated between 30-50%. “Grosso modo” this means that a few millions are really
affected in one way or another, but by what and how is not known in detail. The last
seasonal assessment – post Deyr 2011-2012 – estimated the number of people in “crisis” or
in “emergency” to be 2,340,000, or about 31% of the estimated population.
Sustainability of the project
20. To what extent did the benefits of the project continue after donor funding
ceased?
This is not in question because donor funding has not yet ceased. Donors are prepared to
continue funding because the crisis in Somalia continues to exist, and the information
supplied is very valuable for them and helps them to justify funding. Somalia itself does not
dispose of a comprehensive food security information system and in fact, FSNAU does what
the national government should do. However, through its capacity building component the
project aims at providing lasting benefits extending beyond its termination, whenever that is.
21. What were the major factors which influenced the achievement or non-
achievement of sustainability of the project?
This project is not sustainable in the way projects in general are implemented. The FSNAU
depends for 100% on external funding. Once the donors decide to stop funding, the project
has to finish its activities. The funding for this project is like the fuel for a car. Once the fuel is
finished the car does not move. This made a number of clients say to transfer more activities
to the national structures. This is a valuable option and should also be encouraged but as
long as national Ministries lack financial means, this option is also not viable. FSNAU is a
structure managed by FAO and FAO can benefit in a very positive way from all the lessons
learned. Since FAO is a multi-lateral agency this gained knowledge and experience is
accessible to everybody.
The project is conceived as a temporary response to an emergency situation and therefore
does not have a sustainability objective comparable to development projects. In principle,
the project can stop when the crisis is over. However, the project has a capacity building
component which aims to contribute to ensuring that Somali institutions and partners are
capable of taking up FSNAU’s responsibilities and tasks. The major constraints to effective
capacity building have been the absence of an effective government structure and the
insecurity which forced the FSNAU to operate from Nairobi. Nevertheless, a start has been
made in the relatively stable north-west (Somaliland) and north-east (Puntland) by training
“Focal Points” who have been appointed in several line ministries and agencies. However,
this is not expected to be a very effective and sustainable step in capacity building because
their position is quite isolated, and they are not paid by the government, but by FSNAU.
According to some of FSNAU’s partners in north Somalia there are now better opportunities
to decentralise FSNAU to Somaliland and possibly Puntland, and to intensify capacity
building in order to transfer responsibilities to a government institution. However, they
acknowledge that the political situation is not yet very stable particularly in north-east
Somalia, and that there may be more clarity after the elections later this year. In order to
really build capacity, a fully-formed institution building programme with longer-term technical
support requiring involvement of more partners than FSNAU alone, is required.
Whatever the future strategy for capacity building, there will always be a long-term
dependence on external funding.
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report
FSNAU Food Security and Nutrition Analysis Unit Evaluation Report

Contenu connexe

Similaire à FSNAU Food Security and Nutrition Analysis Unit Evaluation Report

Memorandum dac peer review
Memorandum dac peer review Memorandum dac peer review
Memorandum dac peer review Dr Lendy Spires
 
CV-Berhanu Haileselassie- April 2015
CV-Berhanu Haileselassie- April 2015CV-Berhanu Haileselassie- April 2015
CV-Berhanu Haileselassie- April 2015Berhanu Haileselassie
 
Nutrition guide to data-collection,interpretation,analysis
Nutrition guide to data-collection,interpretation,analysisNutrition guide to data-collection,interpretation,analysis
Nutrition guide to data-collection,interpretation,analysisSM Lalon
 
USAID Tanzania Country Strategic Plan FY 2005 - 2014
USAID   Tanzania Country Strategic Plan FY 2005 - 2014USAID   Tanzania Country Strategic Plan FY 2005 - 2014
USAID Tanzania Country Strategic Plan FY 2005 - 2014Apollo Temu
 
NGO Publications Library
NGO Publications LibraryNGO Publications Library
NGO Publications LibraryDr Lendy Spires
 
Oie e nfinal_arial
Oie e nfinal_arialOie e nfinal_arial
Oie e nfinal_arialkotinagu
 
Sahsp mte-final report-27-feb-2007
Sahsp mte-final report-27-feb-2007Sahsp mte-final report-27-feb-2007
Sahsp mte-final report-27-feb-2007Eddy Mwachenje
 
Tools for planning, costing, implementing and financing scaled-up multisector...
Tools for planning, costing, implementing and financing scaled-up multisector...Tools for planning, costing, implementing and financing scaled-up multisector...
Tools for planning, costing, implementing and financing scaled-up multisector...SUN_Movement
 
AfDB Integrated Safeguards System - Policy Statement and Operational Safeguards
AfDB Integrated Safeguards System - Policy Statement and Operational SafeguardsAfDB Integrated Safeguards System - Policy Statement and Operational Safeguards
AfDB Integrated Safeguards System - Policy Statement and Operational SafeguardsDr Lendy Spires
 
CMC-AH Activity Report_October 2010- September 2012
CMC-AH Activity Report_October 2010- September 2012CMC-AH Activity Report_October 2010- September 2012
CMC-AH Activity Report_October 2010- September 2012Alexandra Brunais
 
Update on “Maziwa Zaidi”—Tanzania dairy value chain development program
Update on “Maziwa Zaidi”—Tanzania dairy value chain development program Update on “Maziwa Zaidi”—Tanzania dairy value chain development program
Update on “Maziwa Zaidi”—Tanzania dairy value chain development program ILRI
 
Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...
Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...
Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...Dr Dev Kambhampati
 
Human energy requirement report 2001 (fao, who & uno)
Human energy requirement report 2001 (fao, who & uno)Human energy requirement report 2001 (fao, who & uno)
Human energy requirement report 2001 (fao, who & uno)Md.Nahian Rahman
 
India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004
India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004
India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004Dr Seán Doolan, MBA
 
21. Tenaw - Emergency SRH ppt-IAWG
21. Tenaw - Emergency SRH ppt-IAWG21. Tenaw - Emergency SRH ppt-IAWG
21. Tenaw - Emergency SRH ppt-IAWGTenaw Bawoke
 

Similaire à FSNAU Food Security and Nutrition Analysis Unit Evaluation Report (20)

Memorandum dac peer review
Memorandum dac peer review Memorandum dac peer review
Memorandum dac peer review
 
Introduction corrected Joke
Introduction corrected JokeIntroduction corrected Joke
Introduction corrected Joke
 
CV-Berhanu Haileselassie- April 2015
CV-Berhanu Haileselassie- April 2015CV-Berhanu Haileselassie- April 2015
CV-Berhanu Haileselassie- April 2015
 
Nutrition guide to data-collection,interpretation,analysis
Nutrition guide to data-collection,interpretation,analysisNutrition guide to data-collection,interpretation,analysis
Nutrition guide to data-collection,interpretation,analysis
 
USAID Tanzania Country Strategic Plan FY 2005 - 2014
USAID   Tanzania Country Strategic Plan FY 2005 - 2014USAID   Tanzania Country Strategic Plan FY 2005 - 2014
USAID Tanzania Country Strategic Plan FY 2005 - 2014
 
CV Template final 1
CV Template final 1CV Template final 1
CV Template final 1
 
NGO Publications Library
NGO Publications LibraryNGO Publications Library
NGO Publications Library
 
Oie e nfinal_arial
Oie e nfinal_arialOie e nfinal_arial
Oie e nfinal_arial
 
Sahsp mte-final report-27-feb-2007
Sahsp mte-final report-27-feb-2007Sahsp mte-final report-27-feb-2007
Sahsp mte-final report-27-feb-2007
 
Tools for planning, costing, implementing and financing scaled-up multisector...
Tools for planning, costing, implementing and financing scaled-up multisector...Tools for planning, costing, implementing and financing scaled-up multisector...
Tools for planning, costing, implementing and financing scaled-up multisector...
 
Prioritizing EU interventions in food insecure regions: The role of IPC
Prioritizing EU interventions in food insecure regions: The role of IPC Prioritizing EU interventions in food insecure regions: The role of IPC
Prioritizing EU interventions in food insecure regions: The role of IPC
 
AfDB Integrated Safeguards System - Policy Statement and Operational Safeguards
AfDB Integrated Safeguards System - Policy Statement and Operational SafeguardsAfDB Integrated Safeguards System - Policy Statement and Operational Safeguards
AfDB Integrated Safeguards System - Policy Statement and Operational Safeguards
 
CMC-AH Activity Report_October 2010- September 2012
CMC-AH Activity Report_October 2010- September 2012CMC-AH Activity Report_October 2010- September 2012
CMC-AH Activity Report_October 2010- September 2012
 
Update on “Maziwa Zaidi”—Tanzania dairy value chain development program
Update on “Maziwa Zaidi”—Tanzania dairy value chain development program Update on “Maziwa Zaidi”—Tanzania dairy value chain development program
Update on “Maziwa Zaidi”—Tanzania dairy value chain development program
 
Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...
Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...
Dr Dev Kambhampati | Feed the Future- West Africa FY 2011-2015 Multi Year Str...
 
8521
85218521
8521
 
Human energy requirement report 2001 (fao, who & uno)
Human energy requirement report 2001 (fao, who & uno)Human energy requirement report 2001 (fao, who & uno)
Human energy requirement report 2001 (fao, who & uno)
 
Compact2025
Compact2025Compact2025
Compact2025
 
India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004
India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004
India Healthcare RCH2 - Sean Doolan & Ruma Tavorath, Oct 2004
 
21. Tenaw - Emergency SRH ppt-IAWG
21. Tenaw - Emergency SRH ppt-IAWG21. Tenaw - Emergency SRH ppt-IAWG
21. Tenaw - Emergency SRH ppt-IAWG
 

Plus de Aart van der heide

Plus de Aart van der heide (19)

ACMKS-brochura final
ACMKS-brochura finalACMKS-brochura final
ACMKS-brochura final
 
ACMKS-brochura final
ACMKS-brochura finalACMKS-brochura final
ACMKS-brochura final
 
ACMKS-brochura final
ACMKS-brochura finalACMKS-brochura final
ACMKS-brochura final
 
ACMKS-brochura final
ACMKS-brochura finalACMKS-brochura final
ACMKS-brochura final
 
diploma WUR
diploma WURdiploma WUR
diploma WUR
 
RAPPORT FINAL UNICEF PASA
RAPPORT FINAL UNICEF PASARAPPORT FINAL UNICEF PASA
RAPPORT FINAL UNICEF PASA
 
Visit to the villages in Waku Kungu in Angola
Visit to the villages in Waku Kungu in AngolaVisit to the villages in Waku Kungu in Angola
Visit to the villages in Waku Kungu in Angola
 
Quelques observations sur le projet
Quelques observations sur le projetQuelques observations sur le projet
Quelques observations sur le projet
 
7. reference HCL
7. reference HCL7. reference HCL
7. reference HCL
 
9. Reference Olga Keita WFP
9. Reference Olga Keita WFP9. Reference Olga Keita WFP
9. Reference Olga Keita WFP
 
reference hamata Energie
reference hamata Energiereference hamata Energie
reference hamata Energie
 
RAPPORT FINAL_EVALUATION MI-PARCOURS_JANVIER 2013
RAPPORT FINAL_EVALUATION MI-PARCOURS_JANVIER 2013RAPPORT FINAL_EVALUATION MI-PARCOURS_JANVIER 2013
RAPPORT FINAL_EVALUATION MI-PARCOURS_JANVIER 2013
 
FINAL REPORT FSN long version 16 may
FINAL REPORT FSN long version 16 mayFINAL REPORT FSN long version 16 may
FINAL REPORT FSN long version 16 may
 
diploma WUR
diploma WURdiploma WUR
diploma WUR
 
Lettre recommandation (4)
Lettre recommandation (4)Lettre recommandation (4)
Lettre recommandation (4)
 
7. reference HCL
7. reference HCL7. reference HCL
7. reference HCL
 
Sumario Executivo Estudo de Base2013_Portugues
Sumario Executivo Estudo de Base2013_PortuguesSumario Executivo Estudo de Base2013_Portugues
Sumario Executivo Estudo de Base2013_Portugues
 
FINAL Projet 2013-311764 Tchad - Rapport Final VFF V1 plus
FINAL Projet 2013-311764 Tchad - Rapport Final VFF V1 plusFINAL Projet 2013-311764 Tchad - Rapport Final VFF V1 plus
FINAL Projet 2013-311764 Tchad - Rapport Final VFF V1 plus
 
Document historique Aart OMAES dernier dubble page
Document historique Aart OMAES dernier dubble pageDocument historique Aart OMAES dernier dubble page
Document historique Aart OMAES dernier dubble page
 

FSNAU Food Security and Nutrition Analysis Unit Evaluation Report

  • 1. Framework Contract EuropeAid: 127054/C/SER/multi - Lot 1 Rural Development Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU) Reference Number 2012/285-027/1 DRAFT - FINAL REPORT April 2012
  • 2. Report submitted by LANDELL MILLS LTD, in Consortium with VetEffecT This report has been prepared at the request and with the financial support of the European Commission. The content of this report is the sole responsibility of Landell Mills and/or VetEffecT and can in no way be taken to reflect the views of the European Commission. KEY DATA Name of Project: Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU) Contractor: DEU to Kenya, Somalia Unit Contracting Authority: 18/03/2012 – 15/06/2012 Start/End Date: Somalia Food Security and Nutrition Analysis Unit (FSNAU) Beneficiary: Nairobi, Kenya Primary Location: Somaliland, Puntland and rest of Somalia Secondary Locations: DEU to Kenya, Somalia Unit DISTRIBUTION LIST Recipient Copies Format Somalia Unit, Delegation of the European Union to the republic of Kenya 10 Hard copy Somalia Unit, Delegation of the European Union to the republic of Kenya 1 electronic copy in PDF Somalia Unit, Delegation of the European Union to the republic of Kenya 1 electronic copy in word ACKNOWLEDGEMENTS The authors wish to thank the EU delegation and stakeholders for the good collaboration for the implementation of the project.
  • 3. Page i Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012 Apr ABBREVIATIONS AND ACRONYMS AGN the nutrition and consumer protection division at FAO CAP (Inter-agency) Consolidated Appeal Process (UN) Candlelight Candlelight for Health, Education & Environment CDC Centre for Disease Control CHF Common Humanitarian Fund CS Central and South Somalia DAC Development Assistance Committee: criteria for evaluation of development assistance of the OECD DEU Delegation of the European Union DFID Department for International Development (UK) DIAL Development Initiative Access Link EC European Commission ENA Emergency Nutrition Assessment EQ Evaluation Question ESAF The Food Security and Agricultural Projects Analysis Unit of FAO EU European Union EWS Early warning System FAO Food and Agriculture Organisation FEWS-Net Food and Early Warning System Network FNS Food and Nutrition Security FS Food Security FSAU Food Security Analysis Unit (1995 – 2009) FSNAU Food Security and Nutrition Analysis Unit (2009 - present) GAM Global Acute Malnutrition HADMA Humanitarian Affairs and Disaster Management Agency of Puntland HAVOYOCO Horn of Africa Voluntary Youth Committee, Somaliland based NGO HDI Human Development Index IASC Inter-agency Standing Committee ICP Integrated Phase Classification ICRC International Committee of the Red Cross IDP Internally Displace People IEC International Education Center IGAD Intergovernmental Authority on Development INGO International Non-Government Organisation IPC Integrated Phase Classification JRC Joint Research Centre of the EC KAP Knowledge, Attitudes, Practices LNGO Local Non-Government Organisation LRRD Linking Relief, Rehabilitation and Development NE North East Somalia or Puntland NERAD National Environmental Research and Disaster Commission of Somaliland NGO Non-Government Organisation NIPHORN Nutrition Information Project for the Horn of Africa NW North West Somalia or Somaliland OCHA Office for the Coordination of Humanitarian Affairs OECD Organisation for Economic Co-operation and Development Oxfam-GB Oxford people for famine Great Britain PENHA Pastoral and Environmental Network in the Horn of Africa PMAC Project Management Advisory Committee (Nairobi based) PoA Plan of Action (FAO) PTF Project Task Force (FAO Rome based) PRRO Protracted Relief and Recovery Operation (WFP) SAM Severe Acute Malnutrition SCF Save the Children Fund SDC Swiss Agency for Development and Cooperation SIDA Swedish International Development Agency SLIMS Somali Livelihood Indicator Monitoring System SMART Standardized Monitoring and Assessment of Transitional Relief
  • 4. Page ii Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012 Apr SSS Somali Support Secretariat SWALIM Somalia Water and Land Information Management TFG Transitional Federal Government TL Team Leader TM Team Member ToR Terms of Reference TTF Technical Task Force UN United Nations UNDP United Nations Development Programme UNHCR United Nations High Commissioner for Refugees UNICEF United Nations Children’s Fund UNSCN United Nations Standing Committee on Nutrition USAID United States Agency for International Cooperation USD United States Dollars VAM Vulnerability and Mapping VETAID Somaliland based NGO part of East Africa Veterinary Aid NGO network WASH Water, sanitation and hygiene WFP World Food Programme WHO World Health Organisation
  • 5. Page iii Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012 Apr CONTENTS ABBREVIATIONS AND ACRONYMS .............................................................................................I EXECUTIVE SUMMARY ................................................................................................................1 1. INTRODUCTION..................................................................................................................7 2. FINDINGS..........................................................................................................................21 3. CONCLUSIONS AND RECOMMENDATIONS ..................................................................33
  • 6. Page iv Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ FinalReport / 2012 Apr
  • 7. Page 1 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr EXECUTIVE SUMMARY 1. Purpose of the Evaluation The objective of the evaluation of phase VI (2009-2013) of the FSNAU is to provide an assessment of the relevance, appropriateness, efficiency, impact and coherence of the FSNAU programme with a view to informing future direction for phase VII (2013-2017). Key questions to be answered are: 1. To what extent has food security and nutrition information and analysis produced by FSNAU been used and for what purposes? 2. To what extent are FSNAU activities appropriate and tailored to local conditions and needs? Are they carried out taking longer-term development and humanitarian scenarios into account? 3. Has FSNAU and FAO created synergies between information system projects of FSNAU and SWALIM? 4. To what extent has FSNAU contributed to improved knowledge on acute and chronic food insecurity and malnutrition and their root causes in Somalia. The evaluation also responds to specific questions that other donors or clients might have. Chapter 2 gives a general introduction about the overall context of the situation in Somalia and the food and nutrition situation in particular. It also explains how the evaluation has been carried out, the methodology used and the formulation of the evaluation questions. The evaluation attempts to give an answer to the four key evaluation questions (EQ 1-4) listed in the Terms of Reference (ToR), the specific questions asked by the FSNAU management (EQ 5-8), the one specific additional question formulated by the DEU Somalia unit (EQ 9) and the specific DAC evaluation criteria (EQ 10-23). In chapter four and five conclusions are drawn and recommendations are given. The evaluators have interviewed the various clients of FSNAU (see annex 2): Somalia government agencies, national NGOs, international NGOs, UN agencies and technical clusters but also all donors. This was done according to a list of specific questions that reflect the specific demands formulated in the ToR (chapter 2.2). 2. Main Analytical Points and Findings 1. The FSNAU provides mainly information of high quality used by a great number of clients varying from local NGOs in Somalia to major donors. This information is used by all clients for various purposes but mainly for donors to justify funding of programmes, for UN agencies and international NGOs to design interventions, and for national NGOs and governmental agencies in the different parts of Somalia – Somaliland, Puntland and CS Somalia – to design specific policies and interventions etc. In general the information is found to be useful by the clients. 2. However, a number of clients mention that the enormous quantity and the complexity of the information make it difficult to find specific information or to understand the information. This has to do with the way information is structured and as such is insufficiently well tailored for specific clients. FSNAU produces a large quantity of information that is understandable for the experts in food security and nutrition but not for NGO or government field workers, general programme managers etc. 3. Existing debates about the concepts of “food security and nutrition” or “food and nutrition security” and the way FSNAU collects and processes all data, refer in the first place to
  • 8. Page 2 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr FSNAU history. FSNAU was created in 2009 from merging FSAU and the Nutrition unit. The fact that FSNAU has a separate Food Security section and a Nutrition section has not been appreciated by all concerned. Concerns exist such as “FSNAU is too FAO”, “FSNAU has a monopoly on data collection and interpretation”, FSNAU is a “Ministry” etc. FSNAU is very aware of these concerns and tries to be as transparent as possible. The question of concepts of “food security and nutrition” or “food and nutrition security” is quite academic. FSNAU collects both food security and nutrition data; nutrition data play an important role in the process of validation. This question needs more “development” and FSNAU is very aware of the need to clarify its role. Optimal client participation in this development should therefore be an option to improve the realisation of FSNAU’s general objective and purpose. 4. A major concern is how to transfer FSNAU tasks to the Somali government as much as possible in an efficient but also effective way given the political situation, security and level of capacities of existing national governmental agencies. Transfer means more sustainability but on the other hand less efficiency and effectiveness given the high turnover of qualified staff in the governmental services etc. 5. FSNAU contributed to improved knowledge on acute and chronic food insecurity and malnutrition but the understanding of their root causes can still be improved. In general it is understood that both forms of food and nutrition insecurity are mainly caused by food but also by water, sanitation and hygiene (WASH) and health related factors but to what degree is not known. To improve understanding, the Health and the WASH clusters request inclusion of more WASH and health related indicators of food and nutrition insecurity. FSNAU is very aware of this concern but indicates that a distinction must be made between “immediate” and “underlying” causes of food insecurity and malnutrition. 6. The question of synergies between the SWALIM and FSNAU information system projects is relevant. The ideal situation should be the full integration of SWALIM and FSNAU information systems with different sections that horizontally cover a number of sections related to food security, food production, livestock, markets, water, hydraulics, irrigation, nutrition etc. and vertically the specific needs for emergency, Linking Relief, Rehabilitation and Development (LRRD) and development information. The specific existing areas of synergies now provide complementary information on water, climate and ecology etc. with specific distinction of in-household water availability and extra- household water availability. 7. The question about whether FSNAU should continue to be the institution that determines the emergency or no-emergency situation in Somalia is understandable but not realistic. There is no alternative for information provision. National governmental agencies do not have the capacity to build an information system such as FSNAU. The perspective to build them up will take many years and certainly higher financial investments. The security and political situation in Somalia is not improving. The arguments used are very reasonable but the reality is that no alternatives are available. Finally it must be recognised that a lot has been achieved under FSNAU and developing a better alternative is not realistic at the present time. 8. Besides the way the FSNAU is organised in sections – food security, nutrition, livelihood – and the way IPC develops its methodology there is a certain concern about the way the workload is divided. Heads of section spend a substantial part of their time on editing the many reports. There is a great need for a professional editor to help the project to restructure the stream of information. The heads of section’s main duty is not editing reports but management of sections and coaching of employees etc.
  • 9. Page 3 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr 3. Main Conclusions I. Use of information? Information produced by FSNAU is used in an optimal way by all clients mainly for project design, justification of new funding, filling the annual Consolidated Appeals, policy development and follow-up of trends. It is also used to declare the situation of emergency or not emergency. II. Appropriateness and if enough tailored to local conditions and needs? As long as the provided information is clear and tailored enough the appropriateness of the provided information is not questioned. However, a number of clients express their concerns about the quantity of information and the high technical level in which it is published. This indicates that despite an appropriate assessment methodology, the information is not sufficiently tailored to local conditions and needs. III. About taking longer-term development and humanitarian scenarios into account? Information produced for emergency scenarios is different from information produced for development activities. Up to now, FSNAU’s priority was to produce information for humanitarian action, based on the food security assessments. Time, budget, security and staff constraints hampered development of data appropriate for support of development programmes. It is expected that this gap will be filled to some extent by the output of the Livelihoods Research & Capacity Building section in the coming years. IV. Synergies between FSNAU and SWALIM been created? These synergies have been created but since SWALIM’s mandate is different from FSNAU’s mandate the synergies are limited. SWALIM does not use “focal points” within national Ministries and therefore the way SWALIM implements capacity building activities also differs from FSNAU. V. Contribution to improved knowledge on acute and chronic food insecurity and malnutrition and their roots? This knowledge is available but given its complexity it has not been presented in an optimal way. Many factors play a role such as: i) FSNAU’s history – a merger of FSAU and the Nutrition project in 2009 – which is also linked to the academic differences between the concepts of food security (FAO and UNICEF); ii) the way “immediate” and “underlying” causes are considered; iii) the fact that food security is concentrating on production, livestock, migration, market developments and - according to a number of clients - less to water, hygiene, health etc. factors. VI. Improvement of capacity building: how, for whom and what? FSNAU has a component of capacity building but both in Somaliland and Puntland no capacity development officer is in charge. Capacity building is directly linked to the sustainability of information collection and analysis. Focal points are used in the Ministries both in Somaliland and in Puntland but despite their training FSNAU experiences a huge staff turnover. VII. Agreement on the issues that FSNAU is the institution that should continue to determine the emergency or no-emergency situation in Somalia? Given the fact that FSNAU is the major institution systematically collecting and analysing national data on food and nutrition in Somalia, and the absence of any alternative this question is not eminent. Adding to that the fact that short or medium term solutions for Somalia are only desirable but not foreseeable, the only way to go forward is that FSNAU is the institution which determines the emergency or no-emergency situation in Somalia.
  • 10. Page 4 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr VIII. Relevance? The relevance is in general recognized. IX. Effectiveness? The effectiveness is in general recognized. X. Efficiency? The efficiency can be improved taking into consideration that certain clients need more “guidance” to understand the information, that information should be more tailored, that information should be diminished in quantity etc. XI. Impact? The positive impact on “direct clients” is recognized, but the impact on “indirect clients” (the needy population) has not, and cannot be verified with certainty. XII. Sustainability? Sustainability of a project such as FSNAU is difficult to achieve. Dependence on funding of a number of donors means that once the donors decide to stop funding the FSNAU cannot continue. In organised countries with a well-functioning administration, FSNAU tasks are carried out by direct line ministries or semi-government agencies. This is not at all the case in Somalia and only partially in the unrecognized countries of Somaliland and Puntland. By means of its capacity building component, the project aims to build a structure that is able to assume FSNAU’s responsibilities in due time. XIII. Coherence? The coherence of the project is not in question. Sections are well-organised and produce the required information. However, the efficiency of the sections can be improved. The heads of section spend a lot of time on editing reports and that time cannot be used for management tasks. The question of methodology improvement is not directly related to the question of coherence but it certainly is related to the transparency of the project such as is experienced by a number of partners. XIV. Added value specific to the European Commission? The added value for the Commission is clear. FSNAU provides information to the DEU and to ECHO. FSNAU is supported by the EU funded JRC. 4. Specific Recommendations. I. Use of information? Try to get a better insight into how information is used and by whom. II. Appropriateness; and if enough tailored to local conditions and needs? Make information simpler and more user friendly, in particular for clients who need more training in how to read, understand and use the information. Make the information more accessible for clients with different backgrounds. III. About taking longer-term and humanitarian scenarios into account? Try to find out in co-operation with clients which information for development is needed, and if and how FSNAU can contribute to generating that information. IV. Synergies between FSNAU and SWALIM been created? Try to find out how capacity development assistance can be developed jointly.
  • 11. Page 5 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr V. Contribution to improved knowledge on acute and chronic food insecurity and malnutrition and their roots? Work jointly with clusters on the question if and how indicators for “immediate” and “underlying” causes can be identified and eventually can be integrated in the surveys in order to identify in a better way the causes of acute and chronic food insecurity and malnutrition. VI. Improvement of capacity building how, for whom and what? Try to find out with SWALIM how this activity can be integrated but only after finding an alternative for the “focal points”. VII. Agreement on the issues that FSNAU is the institution that should continue to determine the emergency or no-emergency situation in Somalia? Try to get full participation of clients – including those in Somalia - in the process of validating the degree of emergency. VIII. Relevance? Continue the policy that clients also own the FSNAU as much as possible: ownership. IX. Effectiveness? Take the concern of national partners regarding the stream of information and the way they understand it seriously into account. Extra training with help of INGO’s is necessary. X. Efficiency? Employ a full time editor for report writing etc. This will help to diminish the workload of the heads of section and give them more time for management, training, coaching, improvement of methodology etc. XI. Impact? Since impact on indirect clients (the food insecure population) is not measurable, try to make a continuously updated inventory of impact of projects and programmes using FSNAU information. XII. Sustainability? No direct recommendations are given but consider capacity building of national institution as a contribution to sustainability. XIII. Coherence? Since the project is coherent enough, improvement of coherence must be sought in the degree of participation of clients. One recommendation is to create a “technical steering group” of partners that have legal influence on the used methodology.
  • 12. Page 6 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr
  • 13. Page 7 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr 1. INTRODUCTION This chapter is an introduction to the project, presenting the general context, the project, the expected results, the methodological approach, the implementation and management arrangements, the beneficiaries, the budget and the donors. It also explains the way the evaluation has been carried out (methodology and programme) and the evaluation questions. It aims to provide sufficient information to understand the conclusions and recommendations worked out in the following chapters. 2.1. Description of the Project. 2.1.1. General Context. Introduction. The Somali people have now suffered the prolonged effects of civil war for more than twenty years. Many attempts have been made by Somalis and the international community to achieve peace and reconciliation in Somalia, but largely the country has remained ungoverned since 1991. In the north of Somalia, two parts i.e. north-east (NE) Somalia or Somaliland and north-west (NW) Somalia or Puntland have a self-declared independence with a local administration – ministries etc. and a government – but Central and South (CS) Somalia are still subject to rival clans and a rising insecurity due to the uprising activities of the Al Shabaab movement. In 2004 the Transitional Federal Government (TFG) was brokered by the Intergovernmental Authority on Development (IGAD) and the broader international community, but it has failed to unite the country and has suffered from internal divisions in the south. It is supported by the Ethiopian military which in itself generates considerable resentment from within Somalia. Mogadishu remains very insecure and the seat of government is still lodged to the south-west in Baidoa. Opposition to the government has gained considerable momentum in recent years represented by both moderate and fundamentalist elements. A political solution to the difficulties in Somalia seems as far off as ever. One particular exception to this situation is in the north-west of the country where there has been relative stability for over ten years and a democratically elected government is well established in Hargeisa. However, Somaliland has yet to gain formal international recognition as an independent state, and its cause and stability has not been helped by the conflict to the south. Neighbouring Somaliland is Puntland, in the north-east of the country, which also declared itself a semi-autonomous state in 1998, but is now loosely associated with the federal system of government in the south. There remains a long-standing and fractious border dispute between Somaliland and Puntland which is unlikely to be resolved in the current context. Food, Nutrition and Livelihood Security in Somalia. Somalia has a total land area of 637,540 square kilometres, a population estimated to range between 7 and 8 million and the longest coast line in Africa. Whilst a semi-arid climate prevails throughout the country, Somalia is characterised by a variety of livelihood zones which are represented in the FSNAU map of Somalia, as shown below:
  • 14. Page 8 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr
  • 15. Page 9 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr The north and inland areas of Somalia are predominantly pastoralist areas combined with fishing activities along the coastal belt. Somalia’s productive economy is predominantly based on pastoralism and agro-pastoralism, so livestock has traditionally been Somalia’s major export. However, in recent years livestock trade has been adversely affected by restrictions on animal exports imposed by Saudi Arabia and other Gulf States. A considerable amount of trade would otherwise take place within and through Somalia since it provides critical sea access to Ethiopia in particular (along with Djibouti). The south, served by the only two perennial rivers of Somalia, the Shabelle and the Juba, has two distinct rainy seasons, the Gu and the Deyr, and is the main area for crop production. However, the average annual cereal production has dropped significantly over the past 20 years since the onset of conflict in the country and there has been an increasing dependence on commercial imports and food aid. Only about 14% of the population is now engaged in cultivation based agriculture. At rural household level, food security relies mainly on subsistence production (livestock and crops) often supplemented by off-farm (sale of bush products) or non-farm activities (labour and petty trade). Remittances from the diaspora (estimated at four to five times the annual international aid budget to Somalia) are an important contributory factor to food security, but more likely benefiting the higher wealth groups than the poor. The south of the country has suffered in recent years from regular floods and drought which have directly impacted on food security in the region. In early 2006, Somalia experienced the worst drought in over a decade which affected pastures and food production. By the end of 2006 the Deyr rains brought relief to drought affected areas, but then excessive river flows caused widespread flooding in southern parts of the country not experienced since 1997-98. Around 255,000 people in the Shabelle and Juba riverine areas were displaced at that time. Now the country is again suffering from failed rains over the past four seasons and facing a deepening drought and water crisis affecting all livelihood zones including pastoral areas. The years of conflict in Somalia has created a situation of protracted and complex emergency which has eroded livelihoods and led to increased vulnerability to food insecurity. In the last years there has been an escalation in the conflict between the Transitional Federal Government (TFG) and insurgents of the opposition in the south of the country. This has resulted in more than 870,000 people being displaced in the last year, many escaping the insecurity in the capital Mogadishu, the worst over the past twenty years. It is also creating an economic crisis throughout much of Somalia characterised by currency devaluation, disrupted trade and market activities and hyperinflation of food and non-food commodities. Within the first six months of 2008, the number of people requiring emergency livelihood and humanitarian support increased 77% from 1.83m to 3.25m affecting an unprecedented 43% of the entire population of the country. An increasing proportion of the population affected are now urban-based. Food insecurity is strongly linked to poverty in Somalia. Access to food and essential services (health, water, sanitation and basic education) remains a major problem for Somalia’s most vulnerable groups: displaced populations, ethnic minorities, subsistence farmers, urban poor and returnees. Throughout central and southern Somalia, typical levels of acute malnutrition in children below the age of five years, outside times of crisis, remain at over 15%, a level that would prompt major emergency humanitarian interventions in other countries. In 2011, Al Shabaab imposed a ban on all foreign aid activities in the area it controls (South-Central Somalia). In the north of Somalia, malnutrition rates are somewhat lower although substantial pockets of high vulnerability are seen in the urban centres, displaced peoples’ camps and areas experiencing extreme environmental degradation. The underlying causes of the chronic nutrition crisis are multi-sectoral, with fundamental factors such as lack of exclusive breastfeeding, early introduction of complementary foods and very limited access to and use of health services creating an underlying vulnerability. This coupled with limited access to safe water and sanitation further predisposes the population to a poor environment for
  • 16. Page 10 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr healthy growth and development which is exacerbated by chronic food insecurity and shocks year after year. Somalia has significantly the highest under five mortality rate and maternal mortality rate in the region at 224 per 1,000 and 1,600 per 10,000 respectively which are key contributory factors to Somalia being ranked 161 out of 163 in the Human Development Index (HDI). 2.1.2. The Project. The Food Security Analysis Unit (FSAU) was first established in 1995 with funding from USAID and the EU and implemented by WFP. Since early 2000, the FSAU project has been managed by FAO, together with a sister Nutrition Surveillance Project. The two projects merged in 2009 in the Food Security and Nutrition Analysis Unit (FSNAU). The current project (Phase 6) runs from March 1st 2009 to February 28th 2013. This four-year phase has a budget of $ 22.75 million; the five expected results are listed below. The overall objective of the project is to ensure that at household level Somali food, nutrition and livelihood security is strengthened, thereby ensuring greater resilience to future “shocks” such as conflict, drought, flood, disease and inflation. The purpose of the project is that a broad range of stakeholders has access to appropriate food, nutrition and livelihood security information for improved emergency response and development planning thereby ensuring that communities, agencies and authorities in Somalia as well as the international aid community are empowered to respond. The project is based in Nairobi in the FAO Somalia office. It employs in its Nairobi office a total number of 35 people and in the whole of Somalia 30 staff1 . The Nairobi office has a Nutrition unit (7 staff members), a Food Security unit (4 staff members), a Livelihood & Research unit (4 staff members) and two support units i.e. the Data processing management unit (10 staff members), Publications sections (2 staff members) and the Operations and Support unit (7 staff members). In Somalia, Somaliland/North-West Somalia and Puntland/North-East Somalia the FSNAU employs 16 field analysts on food security (1 is also liaison officer), 13 field analysts nutrition (of which 5 are also focal points) and 1 capacity development officer. Annex 3 gives an overview of the project’s management structure per section and per area i.e. Nairobi, the CS (= Central South of Somalia), the NE (= North East of Somalia or Puntland) and the NW (= the North West of Somalia or Somaliland)2 and indicates the different titles employed. The Nutrition unit is in charge of assessing the nutrition situation. In annex 12 the list of nutrition monitoring indicators is given and in annex 10 the overview of the annual nutrition surveys. The Food Security unit is in charge of assessing the food security situation. Annex 11 gives the list of the Food Security Monitoring Indicators and annex 9 the annual food security surveys. 1 See in annex 13 the exact numbers of employees in Nairobi and in Somalia. Annex 14 shows the list of employees and their titles. 2 The UN names Somaliland the North-West and Puntland the North-east of Somalia. The central and southern parts of Somalia have the name CS Somalia.
  • 17. Page 11 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr In general there are bi-annual seasonal food security and nutrition assessments – June/July and November/December - and rapid assessments. The section for Livelihood & Research is mainly in charge of research and studies – special subjects – and gives support to the improvement of the assessment methodologies used by the Food Security and Nutrition section. The FSNAU produces an important number of bulletins, studies, updates etc. either through their website www.fsnau.org or in hard copy form. See annexes 7 and 8. The project has been delivering five results – see 2.1.3. - as planned. Given the continuation of civil strife in Somalia, combined with severe food insecurity, malnutrition and famine across several regions of the country, a further phase in support of the FSNAU (phase 7) is being planned for another 4 years beyond February 2013. 2.1.3. Expected Results. The project document mentions five expected results – see below – of which the core results are: timely and relevant information and analysis; improved sector analysis and applied research baseline livelihoods information and analysis, development of an integrated database and capacity building of Somali institutions and partners with special focus on food security, nutrition and livelihoods. Result One: Timely and relevant food security, livelihood and nutrition information and analysis provided on emergency situations Result Two: Increased understanding of opportunities to reduce chronic food, livelihood and nutrition insecurity through improved sector analysis and applied research on underlying causes Result Three: Baseline livelihoods information and analysis generated to inform the design of early response and longer term interventions aimed at improving livelihood security Result Four: Information is further organized, developed and incorporated into an integrated database system and made accessible through managed information systems Result Five: Technical capacity of Somali institutions and partners is strengthened in food security, livelihoods and nutrition monitoring, assessment and analysis N.B. These five results cover in general all activities carried out and all services provided by FSNAU. It must be emphasized that both Early Warning and Emergency preparedness are not included in the five expected results while often requested by clients and partners. 2.1.4. Methodological Approach of the Project. In order to understand the meaning of food security and nutrition or better said “food and nutrition security”, please read annex 15 which explains how to understand the different concepts. To assess the food security situation, FSNAU undertakes the following core activities: 1. Post-Gu and Post-Deyr Seasonal Food Security Assessments and Nutrition Assessments The seasonal food security assessments are conducted immediately after both rainy seasons (the Gu main rainy season from April-June, and the Deyr short rains from October-
  • 18. Page 12 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr December). They cover the entire country and identify areas and socio-economic groups likely to face food security and nutrition problems during the coming period. The nutrition surveys are conducted prior to the seasonal food security assessments and inform on the nutrition and public health situation. These assessments support disaster preparedness and provide food, nutrition and livelihood security analysis and trends information for long term programming. Outputs include: (i) bi-annual ICP food security projections; (ii) bi-annual nutrition situation maps; and (iii) press releases, presentations, special briefs and bi-annual technical series on seasonal assessment results. 2. Rapid Food Security and Nutrition Assessments Rapid assessments are undertaken when a disaster has taken place such as a flood or outbreak of disease, or when the security conditions do not allow regular surveys to be carried out. Outputs include: (i) press releases; (ii) presentations; and (iii) rapid assessment reports. 3. Food, livelihood and nutrition security monitoring and surveillance Monthly or weekly community-level monitoring of the status of agriculture, pastoralism, nutrition, health, water, markets, climate and civil security contributes to baseline analysis and seasonal assessments as well as supporting the early warning function of the FSNAU. Outputs include: (i) quarterly food security and nutrition briefs; (ii) monthly climate and market data updates; (iii) bi-monthly nutrition updates; and (iv) monthly briefings to coordination mechanisms. 4. Baseline Livelihoods Analysis Baseline livelihoods analysis generates understanding of livelihood assets and strategies and helps define longer term interventions aimed at improving livelihoods. 5. Applied thematic research Applied research focuses on gaps in the understanding of underlying causes of food insecurity and malnutrition; it informs longer-term policy and programming, and on support to development/refinement of assessment methodology. Livelihood based analysis A household’s vulnerability to food insecurity may vary considerably according to its livelihood. Therefore, the food security analyses are carried out by livelihood zone. To this end, separate assessments – with separate sampling – are conducted in Internally Displace People (IDP), urban and rural populations, and the rural area is divided into livelihood zones characterized by the predominant livelihood, e.g. pastoral, agro-pastoral or riverine/cultivation (see the map of the FSNAU livelihoods in Somalia on next page). Seasonal nutrition assessments If the security situation allows, the nutrition situation is assessed by means of representative standard household surveys in the rural, urban and IDP populations. The methodology is based on the SMART (Standardized Monitoring and Assessment of Transitional Relief) guidelines3 which also provide the ENA (Emergency Nutrition Assessment) software module for sampling and data processing. Households are selected by Region using two-stage proportionate-to-size cluster sampling. Next, to enable livelihood based analysis, the sample of a livelihood zone is created by clustering the selected and located communities in one and the same livelihood zone. Household questionnaires are used to collect data on household composition, morbidity, mortality, food consumption, food security and access to water. Furthermore, the nutritional status of under-five children and adult women is measured on the basis of internationally accepted anthropometric indices, reference values and cut-off points. From these data, the 3 http://www.smartmethodology.org/
  • 19. Page 13 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr prevalence of acute malnutrition is calculated, with global and severe acute malnutrition (GAM and SAM) as key indicators. In case of time or security constraints, rapid nutrition assessments are carried out instead of standard surveys. They consist mainly of anthropometric data collection and a limited set of questions on morbidity and food security. Due to security restrictions, updated nutrition and mortality data has not been collected in December 2011 in Southern Somalia, apart from the IDP and urban population in Mogadishu. However, indirect information on nutrition trends from health centres and feeding programmes could be collected. Seasonal food security assessments Data on climate, civil insecurity, agriculture, livestock, markets, gender and nutrition of the rural, urban and IDP populations are collected by means of a combination of standard surveys, rapid assessments, monitoring and secondary data collection. Since the Gu 2011 seasonal assessment, representative standard household surveys are used for data collection in the urban areas and IDP camps in areas with good access such as Somaliland and Puntland. Random household selection will take place as described above. In the rural areas, and in urban areas and IDP camps with restricted access due to insecurity, qualitative rapid assessment methods such as focus group discussions and key informant interviews are used. This is complemented by field observations by FSNAU field analysts (livestock condition, crop), continuous monitoring by enumerators (market prices, terms of trade, rainfall, etc.), remote sensing (rainfall, vegetation density), and secondary data (e.g. partner reports)4 . Indicators obtained from these assessments are listed in annex 11. Planning of seasonal assessments An example of the total process of a seasonal food security assessment is presented in annex 19. The assessment starts with a Technical Partner Planning Meeting in Nairobi to discuss participation of partners, timeframe, tools, etc. Partners are UN agencies, INGOs and LNGOs (with an office in Nairobi). Next, there are Regional Planning Workshops in Somalia where local partners (NGOs and local authorities) are informed, their cooperation is confirmed, and team composition and training are prepared. These workshops are led by the regional FSNAU field analysts and the Nairobi-based food security analysts. Depending on a Region’s size, the assessment is carried out by 2-4 teams working simultaneously, each team consisting of a number of enumerators and one supervisor. The latter may be a FSNAU field analyst or food security analyst as well as an experienced person from a UN agency or NGO. Data analysis The objective of the analysis is to describe the current or imminent food security situation. For this purpose, FSNAU uses the Integrated Food Security Phase Classification (IPC) methodology, to the development of which it contributed largely over the past years5 . The IPC is a set of protocols for standardized classification of the severity of food insecurity situations. The IPC consolidates wide-ranging evidence on food insecure people to provide answers to questions as: How severe? Where? How Many? Who? Why? Consensus building by survey teams and experts on outcomes is one of the core characteristics of the IPC process. The IPC is continuously improving its procedures. FSNAU used version 1.0 4 In regions of central and south Somalia where FSNAU has very limited access since the ban of Al Shabaab end 2011, FSNAU field analysts undertake field observations of crop and livestock situation in all livelihoods but, due to insecurity, commodity-level primary data is collected by enumerators and transmitted via teleconference to the FSNAU field analysts. Despite the insecurity, FSNAU continued to receive routine monitoring data through markets and Somali Livelihood Indicator Monitoring System (SLIMS) data points from all regions throughout the assessment periods. 5 The IPC is developed by an innovative multi-agency partnership of eight major UN agencies and international NGOs: CARE International, the Joint Research Centre of the European Commission, FAO, FEWS NET, Oxfam GB, Save the Children (UK&US), WFP that have joined forces to promote a „common language‟ to characterize the food security situation, and more appropriate and effective policies and responses to food insecurity. More information on IPC is available on www.ipcinfo.org.
  • 20. Page 14 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr until the Post Gu 2011 assessment, and used the preliminary version 2.0 for its last assessment of Post Deyr 2011-12. FSNAU organises the IPC process along the following successive steps: 1. Regional Analysis Meetings Data entry starts while the assessments are still on-going. Immediately after the field work, Regional Analysis Meetings are held simultaneously in Hargeysa, Garowe and Mogadishu (situation Post-Deyr assessment 2011/12). Supervised by FSNAU field analysts, the survey teams compile the multitude of evidence from different sources in an ‘evidence based template’; they check the data for quality and consistency with data from remote sensing, regular monthly field monitoring, nutrition surveys, Somali Livelihood Indicator Monitoring System (SLIMS), main market data, and previous trends; they also build technical consensus on the outcomes. 2. The Regional Analysis Meetings, usually in Hargeisa, are immediately followed by a National Analysis Meeting, of two weeks with participation of the assessment teams of all regions, and experts from INGOs, LNGOs, local government, UN agencies, FEWSNET, JRC and FSNAU staff from Nairobi. The regional teams present their outcomes by Region and by livelihood zone to the experts. They adjust the outcomes where appropriate and assign a preliminary IPC class/phase. Next, this outcome is presented to and challenged by the plenary bodies, followed by an ongoing process of revision and challenge, resulting in a final IPC phase based on consensus. On average half a day per Region is allocated to this process. 3. Vetting of the results in Nairobi. This is the final technical check of the analysis results of the National Analysis Meeting by the FSNAU staff in collaboration with its primary technical partners from LNGOs, INGOs and UN agencies. First, the nutrition situation classification is vetted, followed by the vetting of the IPC results, which incorporate nutrition and health indicators. 4. Estimating numbers of affected population by region, district and livelihood zone, based on wealth categories and the deviation of key indicators (e.g. the Minimal Expenditure Basket) from baseline values. 5. Press communications etc. 2.1.5. Implementation and Management Arrangements. Institutional Framework and Coordination The Project Management Advisory Committee (PMAC) meets twice per year to discuss issues related to management, policy, strategy, new project documents, progress reports, and evaluations. The membership includes the donors, UN agencies, the UN Resident Coordinator to Somalia, the FAO Officer-in-Charge, the Chief Technical Adviser to the FSNAU and FSNAU unit managers and the Lead Technical Unit to the Project Technical Task Force. The Project Task Force (PTF), comprising key FAO technical services/divisions such as the Food Security and Agricultural Projects Analysis Unit (ESAF) of FAO and the nutrition and consumer protection division at FAO (AGN), has an advisory function and coordinates technical backstopping and technical missions to the FSNAU (sourced both from within and outside FAO). The PTF also facilitates technical information exchange and supports the FSNAU to identify institutional partners that could support the applied research programme. The PTF meets twice per year in Rome. Somalia Coordination FSNAU works through existing coordination systems to ensure that the project is well integrated into established humanitarian and development planning and information mechanisms for Somalia and engages with all key actors (including donors, other UN agencies, NGOs and government authorities at a decentralised level). FSNAU uses these meetings to share information and analysis, to verify findings, to tailor information and analysis to user needs, and to advocate for appropriate, timely and integrated response.
  • 21. Page 15 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr Furthermore FSNAU contributes to the development of guidelines and strategies which coordination working groups may be tasked to undertake. The FSNAU provides between five to seven informal situation briefings every month through various coordination mechanisms. The committees and working groups to which FSNAU contributes most regularly include: Inter-agency Standing Committee (IASC) Steering Committee; IASC Heads of Clusters Meeting; IASC Cluster Meetings (Nutrition Cluster (FSNAU is co-chair); Health Cluster; Water, Sanitation and Hygiene Cluster; and Food Security Cluster (FAO is co-chair)); Consolidated Appeal Process (CAP) workshops facilitated by OCHA in Somalia and Nairobi; SSS (Somali Support Secretariat) Sectoral Meetings; Zonal Livelihoods Sectoral Committee in the north-west and potential other zones in Somalia. Furthermore, FSNAU contributes to regional level forums such as the Food Security and Nutrition Working Group (Horn and East Africa) of which FSNAU is a member; IASC Heads of Clusters (global level); IPC Technical Steering Committee (global level); Nutrition Cluster Assessment Working Group (global level); and the United Nations Standing Committee on Nutrition (UNSCN) Nutrition in Emergencies Working Group (global level). Other Technical Partnerships: FEWSNET is directly supporting the costs of 24 enumerators for market and climate data, provides lead support on climate and market analysis, and contributes to the overall analysis on food security and livelihoods. SWALIM and FSNAU are both key partners in developing the information base in Somalia on food security, nutrition, livelihoods, water, land and the environment which are closely interconnected. SWALIM is now managing river level monitoring and flood forecasting (contributing to early warning), it generates data on water and land natural resources, including soils, water, river catchment areas, agro-ecological zoning, (contributing to baseline livelihoods asset and zoning analysis). OCHA The relationship with OCHA ensures that food security, nutrition and livelihoods analysis is effectively informing plans, strategies and the allocation of resources in support of an effective and coordinated humanitarian response. UNICEF FSNAU collaborates with UNICEF on the further enhancement of nutrition information, nutrition assessments and methodology, monitoring of immunization coverage, monitoring of water and sanitation indicators, malaria mapping and analysis, micro-nutrient surveys, improved understanding of Knowledge Attitudes and Practices (KAP) to child care and feeding, development of International Education Center (IEC) materials on child care and feeding and the Nutrition Information Project for the Horn of Africa (NIPHORN) which supports local health administrations to expand the capacity for nutrition surveillance in Somalia. Additionally FSNAU has close working relationships with the WFP Vulnerability and Mapping (VAM) unit and CARE which rely significantly on FSNAU data for food aid projections, and the World Bank on economic indicators including recent interest in urban market information. The Nutrition team liaise closely with the Institute of Child Health and Centers for Disease Control (CDC) on technical issues relating to sampling and assessment methodology. The Joint Research Centre (JRC) of the European Commission (EC) trains FSNAU in the application of remote sensing for monitoring rainfall and agricultural vegetation (including pastures), and yield forecasting. FSNAU has a broad network of staff within Somalia in spite of the current security crisis. The field team of food security and nutrition analysts and enumerators represent the backbone of the unit and integral part of the technical team. The field team’s configuration is as follows: • 105 part-time food security enumerators (of which 12 enumerators are funded by FEWS- NET) responsible for collecting primary data on markets, rainfall, key SLIMS indicators and other supplementary information;
  • 22. Page 16 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr • 20 part-time nutrition enumerators responsible for collecting health facility based nutrition data and involvement in assessments on an ad hoc basis; • 16 food security field analysts (including up to four liaison officers) who will review, collate and integrate the data from enumerators, undertake sectoral analysis in their area and facilitate baseline analysis and seasonal assessments. • 14 nutrition field analysts (including five focal points) responsible for the monitoring and reporting of the nutrition situation in their area, conducting rapid assessments and training of partners on nutrition surveillance, micronutrient deficiency, food processing and preservation. N.B. Capacity development officers are both planned in Hargeisa and in Garowe. A new one has to be appointed in Hargeisa. In Garowe no one has been appointed. 2.1.6. Beneficiaries. The project’s general objective indicates to ensure the Somali food, nutrition and livelihood security at the household level. Greater resilience to future “shocks” such as conflict, drought, flood, disease and inflation must be guaranteed. Also a broad range of stakeholders must have access to appropriate food, nutrition and livelihood security information for improved emergency response and development planning thereby ensuring that communities, agencies and authorities in Somalia as well as the international aid community are empowered to respond. The project document mentions that the ultimate beneficiaries to the FSNAU project are the Somali populations who are vulnerable to food and nutrition insecurity. They will benefit from the informed and strategic response of humanitarian and development actors in Somalia. The primary beneficiaries of FSNAU information are the Nairobi-based international community, notably donors, UN agencies, international and Somali NGOs. Particular beneficiaries in the different parts of Somalia are: • Emerging national government structures concerned with nutrition and food security; • Somali Academic Institutions who focus on food security and nutrition education; • Commissions or agencies charged with contingency planning or emergency preparedness (NERAD, HADMA); • Non-Government Organisations and Civil Society Organisations requiring local information for strategic planning and monitoring purposes. 2.1.7. Budget and Donors. The total budget for phase 6 (2009 – 2013 or 4 years) amounts a total of 22,750,000 US$. Major donors are the SIDA, the EU, the USAID, the DFID and minor donors are SDC, Italian Cooperation, UNICEF, UNHCR, OCHA-CHF6 . Result 1 – the timely information and analysis - covers 50% of the budget. The breakdown of the budget is according to the budget proposed and approved of in the project document: Salaries professionals, technical consulting staff etc. 49% 6 6,000,000 US$ by SIDA, 3,900,000 US$ by EU, 5,000,000 US$ by USAID, 3,000,000 by DFID and 1,500,000 US$ by OCHA-CHF, 900,000 US$ by UNICEF, 595,000 US$ by UNHCR, 300,000 US$ by the SDC and 300,000 US$ by the Italian Cooperation.
  • 23. Page 17 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr Travel 23% Contracts 7% Operational expenditure 7% Training, capacity building etc. 5% Rest: (non)-expendable procurements, technical support etc. 9% 2.2. Description of the Evaluation. The methodology used during this evaluation followed traditional procedures. The basis for the evaluation was the Terms of Reference in which the general and specific objectives of the evaluation have been given. The ToR objective lists four key evaluation questions to be answered, and specifies four specific aspects – related to the DAC evaluation criteria - of the project to be assessed (see 2.3.). The required output of the evaluation is a draft evaluation report and a final evaluation report. An inception report is not requested. Although not requested, the evaluation team has written a concise inception report of five pages in which a number of questions have been formulated according to the ToR, including a specifically requested response by the management team of the FSNAU and an additional minor question requested by the DEU Somalia unit evaluation task manager. The first four weeks were mainly used for the inception and for the interviews with all donors, the UN agencies and the main NGOs which are partners in the field and also involved in data collection. The second week the team travelled to Somaliland and Puntland where all Ministries and partners – UN and NGOs – were interviewed including all focal points. The fifth week was used for wrapping up and included a debriefing organized for all stakeholders and clients, whether they had been interviewed or not. The interviews were carried out in a semi-structured way using a list of 16 questions mainly used as a discussion guide and presented below. 1. How do and did you use FSNAU information? 2. What specific information do and did you use? 3. What information do you collect yourself? 4. Does FSNAU cover all fields regarding food and nutrition? 5. If not what should it cover in addition? 6. What information do you miss that FSNAU does not produce? 7. Specific information about political and technical position of FSNAU? 8. Information about the FSNAU methodology? 9. Information about the FSNAU validation quality of information? 10. How can FSNAU improve their performance? 11. Information about the FSNAU structure? 12. Expectations of the FSNAU steering committee? 13. How to improve capacity building, for whom and what? 14. What do you think of the sustainability of FSNAU? 15. Specific recommendations for this evaluation and related to the ToR? 16. Specific wishes of donors and partners? The answers to the questions were worked out in a table presenting the observations and concerns expressed by the interviewees. Annex 16 presents an overview of answers to these questions.
  • 24. Page 18 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr 2.3. Evaluation Questions. A total number of 23 Evaluation Questions have been formulated and answered in chapter 3. The questions are classified according to: the Terms of Reference, FSNAU management, DEU Somalia unit and DAC Criteria. Annex 17 gives the detailed questions and sub- questions to be answered. Questions according to the TOR, the FSNAU management and the DEU. 1. To what extent has food security and nutrition information and analysis produced by FSNAU been used (by decision makers, researchers, implementing agencies and civil society Somali and non-Somali) and for what purposes? 2. To what extent are FSNAU activities appropriate and tailored to local conditions and needs? Are they carried out taking longer-term development and humanitarian scenarios into account? 3. Has FSNAU and FAO created synergies between information system projects of FSNAU and SWALIM? 4. To what extent has FSNAU contributed to improved knowledge on acute and chronic food insecurity and malnutrition and their root causes in Somalia. 5. How can the capacity building component be improved, for whom and what? 6. What has been done well, what not, and how can that be improved? 7. What do partners want in the next phase? 8. How has the information been used by the partners? 9. Do we agree that FSNAU is the institution that should continue to determine the emergency/no-emergency situation of Somalia (a sort of switch on, switch off)7 . Questions according to DAC criteria. Relevance of the project 10. To what extent are the objectives of the project still valid? 11. Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives? 12. Are the activities and outputs of the project consistent with the intended impacts and effects? Effectiveness of the project 13. To what extent were the objectives achieved / are likely to be achieved? 14. What were the major factors influencing the achievement or non-achievement of the objectives? Efficiency of the project 15. Were objectives achieved on time? 16. Was the programme or project implemented in the most efficient way compared to alternatives? Impact of the project 17. What has happened as a result of the project? 18. What real difference has the activity made to the beneficiaries? 19. How many people have been affected? 7 In the last 4 years or more the absent of a Somali state in the south has generated an uncomfortable feeling that FSNAU is replacing an important function of a state to call for assistance or not call for assistance (see Eritrea). How do stakeholders feel on the important political power that we have delegated to FSNAU? Can they manage it? Is the system capable?
  • 25. Page 19 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr Sustainability of the project 20. To what extent did the benefits of the project continue after donor funding ceased? 21. What were the major factors which influenced the achievement or non-achievement of sustainability of the project? Coherence of the project 22. What is the logical, orderly and consistent relation of parts of the project? Added value of the project specific to the Commission 23. Is there added value and if yes, what added value of the project?
  • 26. Page 20 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr
  • 27. Page 21 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr 2. FINDINGS In this chapter evaluation questions and evaluation criteria are analysed and answers are given. The origin of the evaluation questions and evaluation criteria are from the ToR (questions 1 – 5), the FSNAU management team (questions 6-8), the DEU (question 9) and the criteria questions according to the DAC8 as formulated in the ToR (questions 10 – 23). Evaluation Questions and Criteria. According to the Terms of reference, the FSNAU management and the DEU: 1. To what extent has food security and nutrition information and analysis produced by FSNAU been used (by decision makers, researchers, implementing agencies and civil society Somali and no-Somali) and for what purposes? FSNAU produces and analyses general Food Security and Nutrition information on the whole territory of Somalia – Somaliland, Puntland and CS Somalia – in general per livelihood zone, for the category of internally displaced persons (IDP), for the urban and for the rural population on a bi-annual basis. This information is published via the FSNAU website (www.fsnau.org) and a series of technical reports, monthly and quarterly briefs, monthly update bulletins for Nutrition, for Markets, for Climate, baseline reports etc. (see annex 7: FSNAU reports and publications and annex 8: Technical Reports Generated During the Reporting Period phase 6). The users or clients are mainly the donors, the specialized or research agencies, the programme implementing agencies but also the (non)-governmental agencies in Somalia and the civil society in either Somalia or somewhere else. In Somaliland and Puntland the users are all local Ministries, the NERAD, the HADMA and the national NGOs. The information is mainly used by the implementing agencies such as international and Somali NGOs for preparing new project proposals, by donors to justify their support, by UN agencies to be informed but also to formulate their support and fields of special attention, and by the UN and NGO community to launch the yearly consolidated appeals (CAP) as well as allocate funds of the Common Humanitarian Fund (CHF), etc. The information is according to the clients “enough” and “sufficient”; some would like to see more (frequent updates), and to others it is “too much” and not sufficiently structured in a way to understand or to find the information needed. For some clients the great quantity of information makes it that “You cannot see the wood for the trees”. Some clients prefer that food security and/or nutrition data collected by themselves in ‘pockets’ of the livelihood zones or regions/districts, be incorporated in the FSNAU reports, or at least be considered in the analysis of the area. FSNAU argues that information of clients can only be integrated if their methodological validity has been proven. In general the information provided is qualified as very useful to useful. 2. To what extent are FSNAU activities appropriate and tailored to local conditions and needs? Are they carried out taking longer-term development and humanitarian scenarios into account? Appropriateness of activities Because of the long absence of an effective government in Somalia, it was inevitable that the UN assumed responsibility for the coordination of the humanitarian programme, including FSNAU’s core task of providing the information required for decision making on humanitarian intervention programmes. Due to the prevailing civil insecurity in Somalia, the decision to operate FSNAU from Nairobi is appropriate. However, if the situation in 8 Definitions of DAC criteria are given in Annex 10.
  • 28. Page 22 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr Somaliland and Puntland remains stable, decentralisation of (part of the) FSNAU activities to these areas should be considered, as data collection and analysis is likely to benefit from closer contact with the partners (local government and (I)NGOs). It will also enhance opportunities for capacity building. FSNAU uses a wide range of data collection tools that are utilised according to the prevailing conditions. For example, when time or security conditions do not allow standard household surveys to be used, it employs rapid assessments to collect data. Other examples are the use of digital forms to speed up data processing and reporting, and the use of telephones/teleconferencing to transfer data from the field since the ban of Al Shabaab in 2011 made normal operations of FSNAU field staff almost impossible. Some partners have doubts about the reliability of the data transferred by telephone. However, according to FSNAU there is not much reason for concern because the data are cross-validated with data from other sources and obtained in the same period. Some UN agencies and NGOs feel that the data in the FSNAU assessment reports do not provide enough geographical detail to be used as a basis for designing intervention programmes. This is true, but it should be acknowledged that the assessments have not been designed for that purpose. They have been designed to provide a general picture of the severity and magnitude of the food security situation by Region, District or Livelihood zone, and the sample size does not allow analysis at a local level. Implementing agencies themselves have to carry out local in depth investigations, if needed with technical support of partner (I)NGOs. It is conceivable, however, that in future FSNAU, in collaboration with these (I)NGOs, trains the small local NGOs in assessment methodology. FSNAU is already training LNGOs to conduct quality nutrition surveys. In line with the IPC data analysis protocol, FSNAU has made a great effort to involve all of its partners in the data gathering and analysis processes from the initial to the final stage. Although this is generally appreciated by the partners, some concerns were raised by Somali partners. Local government officers said to regret the fact that the final IPC classification step was made in their absence at the vetting in Nairobi. Although they are encouraged by FSNAU to participate in the vetting, and recognize that possible amendments in classification outcomes were relatively rare and comparatively small, it gave them a sense of loss of ownership. To some extent this also seemed to be the case for some Somali NGOs for whom, just like the government officers, there are practical constraints (time, budget for travel and accommodation) to participation in the vetting or even in the National Analysis Meeting in Hargeisa. The outcomes of the food security assessments are frequently and timely published in a range of publications, released as hard copies and as digital files accessible from the FSNAU website. These publications are very much appreciated by most of FSNAU’s clients. However, some of them indicated to have difficulty in finding the right type of information and/or in understanding it. This applies particularly to the NGOs and governmental institutions in north Somalia. By and large, it is concluded that the activities and methods employed by FSNAU are appropriate and tailored to the prevailing conditions and the needs of its clients. Consideration of longer-term development and humanitarian scenarios FSNAU’s activities are focused on food security situation analysis with a view to supporting the design and effectiveness of emergency humanitarian interventions, rather than longer- term development oriented programmes. Therefore, it says it focuses on the immediate causes rather than the underlying causes of food, nutrition and livelihood insecurity. This also explains why FSNAU rightly does not see it as its primary responsibility to carry out certain monitoring activities, e.g. monitoring of water sources, as requested by some partners. The reason for these requests is understandable: FSNAU is the only agency having a country-wide, long-lasting and fairly robust data collection and analysis system. However, besides the above fundamental argument against taking up additional data gathering, the current heavy work load of FSNAU does not allow it to do more work. It would be interesting to see if data collection in support of development planning could be taken up
  • 29. Page 23 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr by other organisations, starting for example in north Somalia and eventually with some guidance from FSNAU. All this does not at all imply that the information generated by FSNAU is not suitable for recovery, rehabilitation or development planning. Although focused on the immediate causes of food insecurity, the assessment reports of FSNAU inevitably also provide insight into the deeper and chronic causes of food insecurity. Moreover, this type of information is also available from the baseline livelihood studies and special research reports of the FSNAU Research and Capacity Building unit. 3. Has FSNAU and FAO created synergies between information system projects of FSNAU and SWALIM? In the past years, SWALIM and FSNAU have cooperated in the following fields:  SWALIM and FSNAU have improved their skills in drought (rainfall) and vegetation monitoring by remote sensing, with technical support of JRC. Jointly with FEWS NET they contribute to the climate sections in publications. It is currently being investigated if remote sensing can also be used for crop forecasting.  FSNAU feeds the flood monitoring system of SWALIM by providing field data on flood impact.  A study initiated by FSNAU on the iodine content of drinking water has been shared with SWALIM  FSNAU financed a study by SWALIM on arable land.  A platform has been created which enables SWALIM and FSNAU to have access to each other’s databases. Although collaboration in some fields has been achieved, both projects acknowledge that it will be difficult to achieve more, given the big difference in nature and mandate of the projects. SWALIM sees a potential role for FSNAU in monitoring water sources but, as explained above, it is difficult for FSNAU to assume this role. According to FSNAU, collaboration is more likely to develop through its Research and Capacity Building section. This would be in support of a suggestion by SWALIM that there may be opportunities for cooperation in capacity/institution building in north Somalia where SWALIM is already supporting three ministries in Somaliland. The current project phases of FSNAU and SWALIM expire almost simultaneously around the end of this year. As suggested by SWALIM, this offers an opportunity to explore opportunities for collaboration in the next phase, and align the project proposals accordingly. In conclusion: there has been more collaboration between FSNAU and SWALIM. There is scope for further collaboration in the use of satellite imaging and possibly in the field of capacity/institution building. 4. To what extent has FSNAU contributed to improve knowledge on acute and chronic food insecurity and malnutrition and their root causes in Somalia? Acute and chronic malnutrition refers in general to a situation in which under-five children are “wasted” according to the relation between their weight and height (acute) and stunted once the growth or height of the child is stunting in relation to its age. Acute or chronic food insecurity has to do with the acute or chronic availability and access to food of good quality and nutritional value etc. FAO and UNICEF use different definitions for the concept of food security (see annex 15) but in general UNICEF is the more nutrition and health related component. However, FSNAU uses the more FAO oriented approach and concept but it must be recognized that the nutrition component has been taken considerably into consideration. It
  • 30. Page 24 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr must be recognized that certain WASH but also health related indicators can be taken into consideration in a more efficient way. We must recognize that FSNAU must take into account a difference between the “immediate” and “underlying “causes of food insecurity and malnutrition. It is their task to assess the “immediate” causes but not the “underlying” causes. The WASH and also the nutrition cluster’s wish to include a number of indicators is justifiable but FSNAU jointly with the two clusters have to sort it out in a positive way how a distinction can be made between the indicators taking into consideration the difference between “immediate” and “underlying” into consideration. It is important that FSNAU and the WASH and Nutrition clusters agree on this issue of the final use of indicators and the weight of each indicator in the process of validation. The question of “root causes” of food insecurity and malnutrition in Somalia is both related to the immediate and underlying causes. It is both food and health related but in what degree is not clear because in “emergency” situations and “normal” situations other factors also must be taken into consideration. 5. How can the capacity building component be improved, for whom and what? Building capacity of the Somali government In a ‘normal country’ the government takes care of the work that is now being done by FSNAU. Indeed, in Africa there are examples of countries that have government institutions conducting food security monitoring and analysis. However, due to the prolonged quasi- absence of a functioning government as a result of civil strife, these institutions do not exist in Somalia. FSNAU has shown to be committed to filling this institutional gap by ‘….. building the technical capacity of Somali institutions where suitable opportunities arise. This builds upon the system of focal points identified in government ministries and FSNAU training initiatives already undertaken in phase V’9 . The Somali institutions benefiting now from this capacity building activity through ‘focal points’ are a number of line ministries and government agencies in the regions of Somalia which have been relatively stable and quiet for a longer period of time, i.e. Somaliland and Puntland. The National Environmental Research and Disaster Preparedness Commission (NERAD) in Somaliland and the Humanitarian Affairs and Disaster Management Agency (HADMA) in Puntland are among the supported agencies. The training of the focal points consists of a computerised training package for self-learning, and training-on-the-job by their participation in the seasonal assessments. According to FSNAU and the ministries, the focal points are employees of the benefiting institutions and they have been carefully selected on the basis of their level of education, experience and motivation. At meetings of the evaluators with ministries, focal points and UN agencies in Somaliland and Puntland, the following emerged:  The governments expressed their desire to accept more responsibility for food security monitoring and analysis, and the need for adequate technical capacity to be build.  The UN agencies feel that it is appropriate now to move more of FSNAU’s work which is currently being done in Nairobi, to north Somalia and build technical capacity at the government level.  There is no clear, unanimous opinion about the modalities of the required food security information structure. Should it be: attached to, or part of a line ministry?; a strengthened NERAD/HADMA?; a new, autonomous institution focusing on food security?; or should it be part of an institution having a wider scope, e.g. a national statistics office from which clients can draw the information they need? Whatever the structure, most interviewees felt that it must be independent/objective and have a strong data management capacity. 9 Quote from the project document.
  • 31. Page 25 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr  The Somaliland and Puntland governments do not have the financial means for the structure and would need to be supported for a long period of time.  Building appropriate technical capacity requires a long-term programme of institution building.  The ministries and focal points were generally happy with the current capacity building activity. However, the focal points indicated the need for face-to-face contact with trainers along with the computerised module for self-training. Although it is clear that FSNAU will still have to do the food security assessments in the foreseeable future, there are opportunities to start capacity building in north Somalia. The focal points approach may be a good way for FSNAU to establish and maintain contacts with the ministries and agencies. Therefore it merits to be maintained for the time being. However, it probably is not an effective and efficient step on the way to institution building because the position of the focal points within the line ministries is too isolated. Moreover, trained people tend to disappear once they have completed their training. It was even noted that some of the focal points were not employed by the ministry before being appointed as a focal point (they worked at the ministry as a volunteer). A comprehensive institution building programme requires a wide variety of inputs which FSNAU cannot deliver alone. Therefore, it requires the involvement of partners as well. A general point of concern in relation to wider government involvement is the risk of the analysis becoming politicised. To minimise this risk, it is recommended to keep the government as well as other partners involved in the conduct of the whole assessment - from the start till the end - and to emphasize the need for rigorous, evidence based analysis. Building capacity of civil society organisations There is a need to train Somali NGOs in the ability to conduct nutrition surveys and food security as well as needs assessments. FSNAU is already involved in such training and this needs to be followed up. However, this training should not and cannot be done by FSNAU alone. Most of the smaller Somali NGOs have partnerships with INGOs or bigger Somali NGOs which, to a greater or lesser extent, (should) already dispose of technical capacity in this field, and therefore should also play an important role in the training programme. 6. What has been done well, what not, and how can that be improved? The concerns and observations (comments) made by a number of clients are not due to a perceived lack of performance of FSNAU, but other reasons were mentioned. The information provided by FSNAU has generally been evaluated as useful for clients. Concerns exist about the reliability of data collected in CS Somalia and transferred by mobile phone. How sure is FSNAU that these data have been collected in a reliable way? Another concern is that results are too general, for instance published for the whole livelihood area, and data from specific pockets within these livelihoods produced by individual agencies were not taken into consideration. A number of clients have expressed the wish to include these data in order to get a more complete picture of the food and nutrition situation in that specific livelihood. According to them, FSNAU does not include these data merely because the validity cannot be controlled. Another concern is the way some information has been collected, among others by FSNAU, and about its validity and objectivity. Main concerns exist about two major themes i.e. 1. The great quantity of information produced. 2. The skills required to read and to understand it. These concerns came mainly from clients working in Somalia in the field but also from a number of Ministries in Somaliland and Puntland. The great quantity of information produced
  • 32. Page 26 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr by FSNAU is a reality and FSNAU should find out how this information can be provided more systematically and how its quantity can be reduced. This makes the need for more tailored information an option: who needs what information? What is information of primary need and what of secondary or tertiary need and for whom? If information is produced in less quantity and more tailored – this needs a detailed needs assessment – technical staff need less time to produce the big stream of information and will make time free for other essential duties such as training and coaching of employees of all clients included FSNAU staff. This is not only a concern of the evaluation team but also of a number of clients. The second concern is directly and indirectly related to the first concern. A number of clients expressed the need of more capacity building in order to understand the FSNAU data better. They were mostly clients based in the field and in the Ministries in Somaliland and Puntland who expressed this need. FSNAU has to find out what they do not understand and how produced information can either be provided in a more tailored way or in a more understandable way. One option is to produce a general manual which explains in detail how to read the FSNAU information10 . This manual could possibly give an introduction to the way FSNAU is organised, what FSNAU is doing, explain the used methodologies, and the way data can be interpreted and used. FSNAU works for a range of clients with different capacities and needs. 7. What do partners want in the next phase? Partners or clients are not requesting big changes or adaptations. A number of changes have been expressed already in the answers to previous evaluation questions but we will summarize them once again.  Integration of WASH and basic health indicators directly related to food security and nutrition in the questionnaires;  Integration of detailed information from specific pockets in a livelihood zone given by clients and after verifying the validity of these data by FSNAU;  Less and more tailored information and in a more structured way;  Capacity building of a number of clients in order to make them able to read and understand the FSNAU information and to conduct quality assessments/surveys;  Evaluate the effectiveness of Focal Points within the Ministries and eventual alternatives;  Review of FSNAU concept of Food Security and Nutrition etc. and more change towards an integral concept of Food and Nutrition Security;  More participation in the improvement of methodologies; 8. How has the information been used by the partners? Most partners use the FSNAU data as a basis for formulating proposals for project funding and for planning interventions. According to some of FSNAU’s partners, donors do not even accept proposals for funding if they are not based on FSNAU data. Organisations that conduct surveys themselves, e.g. the International Committee of the Red Cross ICRC, use the FSNAU data for triangulation. HADMA (Puntland) said to conduct (inter-agency) needs assessments in areas classified by FSNAU as severely food insecure. 10 The evaluators experienced the need of structuring and tailoring the produced information when assessing the ways the various assessments are carried out and in particular the methodology used. It took long time to find out what the different methodologies were and how the assessments are organized. A simple but detailed manual explaining in detail what is done by whom and how etc. makes all procedures more transparent.
  • 33. Page 27 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr 9. Do we agree that FSNAU is the institution that should continue to determine the emergency/no emergency situation of Somalia? Taking the official objective, the purpose and the five expected results seriously into account, it is not FSNAU’s mandate to determine the emergency or no-emergency situation of Somalia. However, the fact is that no other structure exists to do this. The information FSNAU provides and the use of the IPC classification brings FSNAU in a position to be the only structure to be able to declare emergency/no-emergency. The overall objective mentions to ensure that the “Somali food, nutrition and livelihood security at the household level is strengthened thereby ensuring greater resilience to future shocks” and the purpose is “that a broad range of stakeholders have access to appropriate food, nutrition and livelihood security information for improved emergency response and development planning”. The five expected results will mention mainly “timely and relevant information for increased understanding of opportunities”, ”through good sector analysis and research to reduce chronic food, livelihood and nutrition insecurity”, “information and analysis to inform the design of early response and longer term development”, the organisation of a professional data system” and “capacity strengthening or building of Somali institutions and partners”. Therefore the answer to this question is easy: Yes. FSNAU is the only capable institution in the absence of a qualified government structure. Therefore FSNAU must continue to assess and classify the food security situation, involving all of its partners as much as possible. DAC criteria: Relevance of the project 10. To what extent are the objectives of the project still valid? Evaluation question 9 explains the validity of the objective and purpose of the project in such a way that it clarifies the duties of FSNAU and also its mandate. It is up to the clients - donors, Somalia institution, UN agencies, NGO’s etc. – to determine what institution should be able to declare the state of emergency or not. From a technical point of view FSNAU is certainly competent to do this but from a more political point of view this is questionable because it gives FSNAU a very heavy responsibility. In general, the overall objective and the project purpose are still valid because the situation in Somalia has not changed or - better said - improved over the last years. 11. Are the activities and outputs of the programme consistent with the over-all goal and the attainment of its objectives? The main activities concern data collection, analysis and publication. This is also what has been mentioned as the general objective and the main purpose. In the five expected results it has been worked out how the collected information and analysis must be used and for what purposes, including the capacity building and the general support FSNAU is giving to the clients. There is no inconsistency between the activities and outputs on the one hand and the overall goal and attainment of its objectives on the other. The only concern is how to make information more accessible and more simple, and also to increase the transparency of the methodological approach of the project. Generally this consistency exists. The activities are all geared to generating quality information (output) relevant to emergency response planning by programme implementing partners and donors. However, some partners (particularly Somali NGOs and government) would like the reports to be written in a more comprehensible language.
  • 34. Page 28 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr 12. Are the activities and outputs of the project consistent with the intended impacts and effects? Yes. By and large, the clients of FSNAU are satisfied with the information provided by FSNAU, and they actually use that information as a basis for designing emergency interventions to strengthen the food security situation of Somali households. The extent to which the interventions have actually led to improved household food security cannot be determined with certainty, but it is very likely that they have been beneficial. Obviously, the actual food security situation of households does not depend on FSNAU food security analysis alone, but also on factors beyond control of FSNAU. Effectiveness of the project 13. To what extent were the objectives achieved or are likely to be achieved? The general objective of the project is to ensure the food, livelihood and nutrition security in Somalia etc., but it has already been explained in evaluation question 12 that achievement of the overall objective is difficult to measure. It is also explained why. This question can be answered more easily for the purpose of the project i.e. “access by stakeholders to data for improved emergency response and development planning”. This is fully achieved but some recommendations have been given on how to improve but also to complete the information. It concerns mainly the wishes of the WASH and Nutrition clusters to include extra indicators related to the “immediate causes of malnutrition”. The issue of the broad range of publications has already been mentioned but also the need to structure the information in a better and more logical way. In order to prevent clients from saying “You cannot see the wood for the trees”, the FSNAU must analyse the following question very seriously: “How can we simplify the information in a more structured and understandable way?”. Maybe the need for a professional editor must be taken seriously into consideration in order to release the heads of sections from having to edit the great number of reports etc. they produce, and to help them to structure and simplify the number of publications. 14. What were the major factors influencing the achievement or non-achievement of the objectives? The factors influencing the achievement of the objectives and purposes are:  Sufficient means and budget;  Well qualified technical staff;  Good cooperation with clients: UN-agencies, INGO’s, Somali government staff and donors;  Good technical support from the Rome based Technical Task Force TTF and other UN agencies and NGOs;  Good management;  Enough confidence from donors’ side;  Involvement of all clients in the validation procedures of data collection and analysis;  Factors influencing non-achievement: extreme civil insecurity, leading to extremely restricted access to the field for data collection.
  • 35. Page 29 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr Efficiency of the project 15. Were objectives achieved on time? There is no clear time path or planning for achievement of objectives and purposes. The project exists since 1995 and the nutrition unit was merged into the FSAU in 2009. There have not been serious concerns or complains about non- achievement of the objectives, purposes or expected results beside that some information comes rather late. The project has no predetermined end point. Its services are required as long as food insecurity lasts, and this depends to a great extent on the civil insecurity that prevails in the country. FSNAU regularly updates its data and information to allow the humanitarian community to adjust its interventions to the prevailing conditions. Except from being sometimes late – this could not be verified by the evaluators – the data and information are generally updated in time. 16. Was the programme or project implemented in the most efficient way compared to alternatives? Alternatives are not known and other structures implementing such a project for Somalia do not exist. It has been mentioned already that more efficiency can be achieved if more structured and more tailor-made information will be provided. There is a need for a simple manual introducing outsiders in a more efficient way to the project, its organisation, its methodology, the frequency of all assessments etc. etc. The need for an editor has also been mentioned to release heads of sections but also to help FSNAU to systematise their different types of information. Impact of the project 17. What has happened as a result of the project? As a result of the project, donors were able to allocate their funds in a rational way, and programme implementing organisations were able to substantiate funding proposals and identify relevant and appropriate interventions. Out of the five expected results mentioned in chapter 2.1.3., the first four have been achieved “grosso modo”. The last result i.e. the capacity building should be given more consideration mainly for national government partners and for NGOs mainly working in the field. SWALIM applies a different way of capacity building, not based on Focal Points. FSNAU has to assess this and alternative ways jointly with SWALIM and other potential partners. 18. What real difference has the activity made to the beneficiaries? It has already been explained that the impact on the indirect beneficiaries – IDP’s, vulnerable families, drought victims, malnourished children – can hardly be measured. For the direct beneficiaries (donors, agencies etc.) the impact has been quite positive. 19. How many people have been affected? What is the meaning of the term ‘affected people’ and affected by what? Somalia suffers from a multitude of problems which affect people: civil war, insecurity, drought, floods, lack of facilities such as health and education, no governance etc. The total population of the Somalia area is not known, the numbers of IDP’s can only be estimated, the number of
  • 36. Page 30 Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)/ LML - VetEffecT/ Final Report / 2012 Apr refugees living in neighbouring countries can be calculated, the number of Somalis in the diaspora can also be calculated. The number of people affected differs from period to period and is estimated between 30-50%. “Grosso modo” this means that a few millions are really affected in one way or another, but by what and how is not known in detail. The last seasonal assessment – post Deyr 2011-2012 – estimated the number of people in “crisis” or in “emergency” to be 2,340,000, or about 31% of the estimated population. Sustainability of the project 20. To what extent did the benefits of the project continue after donor funding ceased? This is not in question because donor funding has not yet ceased. Donors are prepared to continue funding because the crisis in Somalia continues to exist, and the information supplied is very valuable for them and helps them to justify funding. Somalia itself does not dispose of a comprehensive food security information system and in fact, FSNAU does what the national government should do. However, through its capacity building component the project aims at providing lasting benefits extending beyond its termination, whenever that is. 21. What were the major factors which influenced the achievement or non- achievement of sustainability of the project? This project is not sustainable in the way projects in general are implemented. The FSNAU depends for 100% on external funding. Once the donors decide to stop funding, the project has to finish its activities. The funding for this project is like the fuel for a car. Once the fuel is finished the car does not move. This made a number of clients say to transfer more activities to the national structures. This is a valuable option and should also be encouraged but as long as national Ministries lack financial means, this option is also not viable. FSNAU is a structure managed by FAO and FAO can benefit in a very positive way from all the lessons learned. Since FAO is a multi-lateral agency this gained knowledge and experience is accessible to everybody. The project is conceived as a temporary response to an emergency situation and therefore does not have a sustainability objective comparable to development projects. In principle, the project can stop when the crisis is over. However, the project has a capacity building component which aims to contribute to ensuring that Somali institutions and partners are capable of taking up FSNAU’s responsibilities and tasks. The major constraints to effective capacity building have been the absence of an effective government structure and the insecurity which forced the FSNAU to operate from Nairobi. Nevertheless, a start has been made in the relatively stable north-west (Somaliland) and north-east (Puntland) by training “Focal Points” who have been appointed in several line ministries and agencies. However, this is not expected to be a very effective and sustainable step in capacity building because their position is quite isolated, and they are not paid by the government, but by FSNAU. According to some of FSNAU’s partners in north Somalia there are now better opportunities to decentralise FSNAU to Somaliland and possibly Puntland, and to intensify capacity building in order to transfer responsibilities to a government institution. However, they acknowledge that the political situation is not yet very stable particularly in north-east Somalia, and that there may be more clarity after the elections later this year. In order to really build capacity, a fully-formed institution building programme with longer-term technical support requiring involvement of more partners than FSNAU alone, is required. Whatever the future strategy for capacity building, there will always be a long-term dependence on external funding.