Please see the RFP document with Appendix 2 for full details on the requirements of the assignment and the requirements of the proposal submission: Evaluation of the ICRC’s Approach to Partnering for Sustainable Humanitarian Impact | ALNAP
About this evaluation
The ICRC is commissioning an independent external evaluation of its approach to Partnering for Sustainable Humanitarian Impact (SHI) strategy from 2019 until the start of the evaluation. SHI initiatives and the emerging Multi-Year Multi-Partner (MYMP) portfolio of projects, which are only a component of SHI, are ongoing as part of the ICRC’s current strategy, whereas sustainability is defined mainly by a focus on strengthening existing structures and local capacities (self-sufficiency) and resilience to shocks.
The primary objective of this evaluation is to assess best practices and lessons learned regarding effective partnerships across the SHI initiatives and projects. The evaluation seeks to provide a set of actionable and strategic recommendations for future decisions related to partnering for SHI. Additionally, the evaluation aims to explore the types of partnerships that contribute to sustainability, scalability, and capitalize on the added value of the ICRC, as well as identify the factors that enable or hinder success in such partnerships for SHI initiatives and projects. Lastly, the evaluation will analyze how the SHI initiatives and projects assess the maturity level and readiness of their partners to assume responsibility.
The ICRC’s Institutional Strategy 2019–2022[1] coined the term “Sustainable Humanitarian Impact” as one of its five Strategic Orientations (SO). The rationale behind this Orientation is the need to ensure the relevance, effectiveness, and impact of the ICRC’s work with populations affected by protracted conflicts and chronic situations of violence. The Institutional strategy was launched in 2019 to meet the ICRC’s Strategic Orientation 2 and 3 of its institutional strategy 2019-2024.[2]
The essence of what SHI aims to achieve was re-affirmed and captured in the 2022-2024 Strategy Implementation Roadmap in the following terms:
‘People affected by protracted conflict, exacerbated by multiple hazards including climate change, pandemics and socio-economic exclusion, experience the ICRC as an accessible, inclusive and accountable organization that is effectively contributing to strengthening their resilience, mitigating their emergency needs, and supporting them to recover their livelihoods and building pathways to self-sufficiency.’
The Strategy Implementation Roadmap reaffirms the central role of partnerships with development actors and the Movement for building SHI.
The ICRC's Partnering Strategy with relevant actors is driven by a desire to achieve greater results and impact on a larger scale, in line with our Accountability to Affected People. By partnering with others, we aim to reach a larger proportion of people affected by armed conflict and other forms of violence than we would be able to reach on our own. Additionally, partnering offers opportunities for the ICRC and its partners to achieve greater relevance, effectiveness, coherence, efficiency, and sustainability. The engagement of numerous stakeholders with various roles and responsibilities, as well as various degrees of partnerships across numerous initiatives and projects, adds to the complexity of Partnering for Sustainable Humanitarian Impact (SHI). This presents a challenge for the evaluation team, which must take these factors into account and deal with these complications in order to offer a thorough and accurate review of the ICRC's partnering strategy.
[1] The ICRC strategy was amended to extend until 2024
[2]ICRC Strategy 2019–2024 | International Committee of the Red Cross
Description of the theory of change or logic model
By consulting with the ICRC's relevant stakeholders and unit heads, the evaluation team will reconstruct the theory of change for Partnering for Sustainable Humanitarian Impact.
The evaluation team will need to underline the partnerships that exist throughout the SHI projects, as well as the planned products, the expectations, external factors, and outcomes, as well as the pathways that allow the activities to achieve the anticipated outcomes. The contextual elements that can support or hinder the sustainability of the anticipated outputs and results must also be considered in the theory of change.
Context of implementing the intervention
The evaluation is taking place as the ICRC's new institutional strategy for 2024–2029 is being formulated and the current Institutional Strategy 2022–2024 which is going to conclude soon after the end of this evaluation. The evaluation's findings and recommendations will be considered while developing the strategy.
Evaluation Purpose
In recent years, there has been a growing demand from donors for international organizations to establish more equitable and strategic partnerships. This includes investing in capacity strengthening and providing multi-year funding where possible to support longer-term partnerships.[1]
With this context in mind, the purpose of the evaluation is to provide formative support to the ICRC's strategic objective on Partnering for Sustainable Humanitarian Impact (SHI), with a particular focus on strengthening the partnering approach and its effectiveness including in terms of successful exits or handovers. The evaluation is intended for learning purposes, rather than accountability, and the recommendations will be used to inform clarity of intent for ongoing SHI initiatives and future projects. Ultimately, the findings and recommendations will also support the internal revision of the SHI objectives within the new institutional strategy and key definitions such as partnerships, exit or handover strategy, sustainability, and their implications for the ICRC.
Overall, the evaluation offers the ICRC a possibility to enhance its SHI approach, better serve affected communities, generate more strategic partnership, and meet evolving stakeholders’ concerns, including those of donors. The evaluation will provide insights into the ICRC’s strategic objectives on Partnerships for SHI, the unique advantages and resources the ICRC brings to its partnerships, the effectiveness of the monitoring systems used to measure the effectiveness of partnerships for SHI, the challenges faced in building, maintaining and ending effective partnerships, and the measures taken to ensure mid-term to longer-term sustainability for the SHI projects and understand the ICRC’s mid to longer presence in protracted crises.
Finally, the evaluation aims to offer valuable insights on how the ICRC SHI team develops, manages, and reviews its partnerships in fragile and conflict-affected areas, including protracted crises. In addition, the evaluation will unpack how the ‘Do No Harm’ (DNH) and conflict (CA) are used and integrated across the SHI partnership processes. Thereafter, the evaluation can conclude on whether the integration of these tools (DNA and CA) have generated the expected outcomes or not and underline to what extent the SHI’s objectives of generating sustainable impact have been met or not.
Objective/s
The objectives of the evaluation are:
- The evaluation seeks to draw evidence-based findings on the partnerships for SHI that have yielded the expected results that are sustainable and scalable in protracted crises and conflicts. The evaluation will help underline what success looks like for partnerships for SHI.
- The evaluation will unpack what partnerships for SHI and enabling external factors are contingent to ICRC’s exit in protracted contexts. The evaluation will draw a set of actionable recommendations for upcoming SHI projects and initiatives. These recommendations will be based on a thorough understanding of the current state of partnerships, including what has worked well and what could be improved. The recommendations will be practical and strategic, aimed at enhancing the effectiveness, relevance, coherence, effectiveness, and sustainability of the outcomes and desired impact. Ultimately, the evaluation will help the ICRC better serve affected populations and meet the evolving expectations of its stakeholders.
[1]ODI Rethinking capacity and complementarity for a more local humanitarian action
Scope
The evaluation will cover the period from January 2019 to present day, although some projects will still be ongoing during the evaluation process.
The evaluation team will focus only on the ‘SHI as a portfolio of projects’ and not on the ‘SHI as a mindset[1]’ for this evaluation, whereas sustainability is defined mainly by a focus on strengthening existing structures and local capacities (self-sufficiency) and resilience to shocks. The evaluation will focus on the initiatives and projects rather than the mindset of the SHI roadmap.
The geographic scope of the evaluation will be determined by the evaluation team in consultation with the ICRC evaluation manager and the evaluation advisory committee assigned to this exercise. A shortlist of relevant countries has been identified for the evaluation team to consider and select from, but the final decision will be based on the team's suggested approach in the inception report, which will provide a clear rationale for selecting specific countries for a deep dive.
The evaluation team will need to consider several factors when selecting the countries to focus on, including the presence and scope of SHI initiatives and projects, the level of engagement with partners, the diversity of contexts and challenges, and the potential for learning and knowledge sharing. The team will need to justify the rationale for their selection, demonstrating how it aligns with the evaluation's overall objectives and scope.
The geographic scope of the evaluation is a critical component that will enable the team to provide a nuanced and contextualized analysis of the ICRC's partnering approach. Through a deep dive in selected countries, where there will be a combination of several types of SHI projects and initiatives, the evaluation team will be able to identify specific opportunities and challenges that can inform the ICRC's future partnerships and initiatives.
Additionally, the evaluation will need to dive into all the different thematic units and projects at the inception phase and narrow down the geographies and thematics to carry out the field data collection work. The suggested units are the following: IDP; Detention; WATHAB; ECOSEC; Health; Resource Mobilization, particularly the Development Actors CoE,, the Multi-Year Multi Partners (MYMP) projects. The evaluation team will also have to review and consult stakeholders from the Movement. Some key documents on the Movement – such as the Senior Leadership Informal Cell on Covid-19; the Collective Impact/SASM Review process that highlights the Value Proposition for the movement and the SHI Partnership Council on a Movement approach in protracted conflict. All relevant documents will be submitted once the team has been selected.
The evaluation team will also be encouraged to assess ongoing initiatives within the ICRC that may fall outside the MYMP and determine whether the partnerships established in these initiatives meet the criteria for sustainable humanitarian impact. This broader assessment will provide a more comprehensive understanding of the ICRC's partnering approach and help identify potential opportunities for leveraging existing partnerships to achieve greater impact. Some projects that could potentially be part of the scoping exercise include:
- Iraq where there are two projects ‘- Irbil hospital and Mosul Hospital-
- Afghanistan – Kandahar hospital
- Honduras- school constructions
- Colombia within the health in danger sector
- WATHab in Gaza with the rehabilitation water system project.
- South Sudan Hospital –
- Peshawar Hospital
- One in West Africa- yet to be identified
Intended users
- The SO-relevant offices – Strategic Advisers, OBA, RBM, MYMP, WatHab, ECOSEC
- The relevant metiers – 2020-23 WatHab Unit, 2020-23 health unit, Resource Mobilization, particularly the DA and NFM CoE, head of partnerships at MoV Movement division
- The Partnership Council members for SHI.
- Heads of delegations across all regions.
[1] See annex for chart precising the two definitions of ‘SHI as a portfolio of projects’ and ‘SHI as a mindset’
Key stakeholders
The evaluation will include stakeholders from the ICRC HQ and the Field, international financing partners such as the World Bank, AfD, amongst others; the Red Cross and Red Crescent Movement, traditional donors (when relevant), and, where possible, national, and local authorities, local civil society representatives and the affected populations. The evaluation team will present a clear sampling methodology at the inception report phase based on the list of stakeholders provided by the SHI team. The sampling will be discussed with the Evaluation Advisory Group and approved by the Evaluation Office.
Evaluation criteria and questions
The evaluation will use a combination of DAC and ALNAP criterion as Relevance, Strategic Coherence, Effectiveness, Sustainability and Coordination amongst partners, as well as the added value that the ICRC brings to these SHI projects and initiatives in these protracted crises and conflict areas.
Relevance
- To what extent does the ICRC’s partnership for SHI address the needs and priorities of affected populations in the protracted crises?
Strategic Coherence
- How does the partnerships for SHI align with the ICRC’s strategic goals and objectives in protracted crises? How does it contribute to the organization’s positioning in these contexts?
Strategic added value
- What unique strategic advantages do the different types of partnerships for SHI offer compared to the other humanitarian and development actors operating in the same geographies?
Effectiveness
- To what extent have the partnerships for SHI achieved their intended results and outcomes?
- How do these results and outcomes contribute to the ICRC’s overall goals and objectives of SHI in protracted crises?
- How effective are the monitoring systems used by the ICRC to measure the effectiveness of partnerships for SHI, and how can these systems be improved to provide better insights to the impact of these partnerships?
Sustainability
- How is sustainability integrated into the planning and implementation of partnerships for SHI?
- What measures are being taken to ensure long-term viability and impact of these SHI projects?
- What are the implications for the ICRC’s SHI projects in the mid-term to longer-term in these protracted crises?
Evaluation approach
Overall design, methodology
The evaluation team will propose a best suited methodology to respond to the objectives of this evaluation.
The evaluation should utilize both qualitative and quantitative tools to assess the wide-ranging thematics, métiers[1], and units included in the SHI. The evaluation office suggests dedicating a good part of the evaluation to a thorough desk review to draw preliminary findings and identify any data gaps that will need to be collected during the field visits. The evaluation team could select two or three case studies to collect best practices and lessons learned on partnership models, stakeholder engagement, sustainability, and exit strategies enablers. These case studies could be chosen from different countries and contexts to facilitate comparative analysis.The evaluation team could refine the case study locations and questions to ensure that the most relevant and informative data is collected.
While the evaluation will not include a cost-analysis, this decision is based on the complexity of the initiatives and projects that are funded through different mechanisms. The cost-effectiveness analysis would not yield conclusive results that could be used to draw conclusions on the evaluative questions above.
Proposed phased approach that can be adapted by the evaluation team.
Inception phase
The evaluation team will carry out a thorough desk review that will include:
To generate a comprehensive partnership chart, the evaluation team will first conduct a mapping of the different types of partnerships across SHI projects and initiatives. This chart will help classify partnerships according to their style and identify the keys to success for each. The team will also review SHI initiative and project documents across multiple MYMPs and thematic units, as well as among Movement[2] partners. The SHI is composed of projects and initiatives that go beyond MYMP, from WatHab and ECOSEC as well as the Movement.
This thorough review will provide a better understanding of the partnerships and their impact. Additionally, the team will conduct a context and situational analysis for the countries where they conduct field data collection to gain a deeper understanding of the local factors that may affect the partnerships and their sustainability.
Field Data Collection phase
The evaluation team could employ a mixed-methods approach to data collection, including key stakeholder interviews, group discussions, and field observations at various sites and projects.
In addition, the team could carry out research on case studies to gather in-depth information on best practices and lessons learned related to partnership models, stakeholder engagement, sustainability, and exit strategies enablers.
If deemed necessary and relevant by the evaluation team, surveys may also be used to gather data from a wider range of stakeholders. If surveys are used, stakeholders will be provided with clear instructions and reminders to encourage participation during a specific period and ensuring confidentiality is respected.
Analysis and writing
The evaluation team will employ a rigorous process of data triangulation and coding to derive key findings that directly address the evaluative questions. The team will analyse the data to uncover valuable lessons learned, draw key conclusions, and develop actionable recommendations for the future SHI strategy.
Risks, limitations
The evaluation team will complement or replace this list of risks and limitations during the inception phase. The evaluation team will also offer solutions on how to mitigate them.
Validity and reliability of research and findings
Validity and reliability of the information gathered will be ensured mainly by ensuring consistency in the analytical approach and by triangulating the same information from various data sources. Not all evaluation questions will be answered in a clear-cut fashion. For some questions there may be clear, quantitative or fact-based answers, while many others may reflect a range of views and perceptions by project participants, beneficiaries, and observers. In cases where there are contradictory findings from different informants or different data sources, the whole range of opinions will be reflected in the final report.
Data Sources
The evaluation team will draw from a diverse range of data sources, including documentary information, monitoring records, key informant interviews, group discussions, surveys, secondary sources, and internal and external analyses and studies. This approach will ensure that data is gathered from various projects, initiatives, strategies, stakeholders, and partners, facilitating sound triangulation of information. In cases where triangulation is not possible, but the findings are highly relevant, the evaluation team will indicate the types of sources used to draw these conclusions.
[1] Métiers are the different specializations found within the ICRC such as engineers, cash specialists, doctors etc.
[2] The Movement is the way the ICRC describes the National Societies of the Red Cross. These National Societies have their own budget and their own strategy but collaborate with the ICRC and the IFRC on some initiatives.
Feedback and validation of findings
Regular feedback will be ensured by the establishment of an Evaluation Advisory of HQ and field stakeholders, who will follow the evaluation process and provide regular feedback on the process laid out in this inception report and the draft report. A preliminary findings workshop can be organized before the final report is submitted. This workshop will ensure ‘buy in’ from all relevant stakeholders, and that the recommendations are actionable and realistic.
Access to external stakeholders
Access to stakeholders may be limited or sensitive in the context of the ICRC's work in protracted crises and conflict zones. It is rare for the ICRC to permit external evaluators to engage directly with stakeholders in these settings. However, the independent evaluation team will ensure that they adopt a "do no harm" approach and reassure stakeholders that all information shared will be kept strictly confidential, and no attribution of comments will be made in the final report. The team will also work closely with the ICRC to identify alternative approaches for collecting information and engaging with stakeholders, such as through key informants or group discussions. Any limitations on access to stakeholders will be noted in the final report and the evaluation team will take steps to mitigate the impact of these limitations on the overall evaluation findings.
Ethical considerations and safeguarding
Evaluators are required to adhere to international best practices and standards in evaluation. It explicitly requires evaluators to abide by the Professional Standards for Protection Work; the ICRC's Code of Conduct; the ICRC's Code of Ethics for Procurement; and the ICRC Rules on Personal Data Protection.
The evaluation design and implementation must apply the ICRC's guiding principles and approach on Accountability to Affected People.
As a strategic evaluation rather than an evaluation of the impact on communities, no affected people will contribute directly to the evaluation, and therefore formal Ethical Review Board approval is not required. While the evaluation will not directly consult populations affected by conflict, the design and implementation should nevertheless consider and apply the ICRC's guiding principles and approach on Accountability to Affected People. However, the evaluation should not entirely exclude the views and experiences of community members if possible, and where there is secondary data, it will be made to the evaluation team (for example, perception surveys, service satisfaction surveys, or other reports document the views of community members).
The confidentiality of participating respondents will be protected through anonymization. All primary data plus any sensitive and internal ICRC documentation will be subject to appropriate data protection and management practices. No data or documentation will be shared outside of the contracting supplier without the express permission of the ICRC.
Management of the evaluation
The evaluation will be managed by the Evaluation Office and the ICRC Partnership Coordinator. An Evaluation Advisory Group will accompany the process and provide expert advice and feedback at key stages of the evaluation. This engagement is particularly important but not limited to the main products of the evaluation i.e. the TOR, draft inception report, and draft evaluation report. The membership of the advisory group is internal to the ICRC. The protocol for the advisory group is set out in its dedicated TOR.
Evaluation Advisory Group (EAG)
An Evaluation Advisory Group will accompany the evaluation, providing their expert advice and feedback on key stages. The members of the EAG may be included in the initial onboarding process for the evaluation team. The EAG will be consulted for the inception phase presentation/ discussion on the plan for data collection and analysis; and for the presentation of the final report findings. The membership of the advisory group is internal to the ICRC and the composition will be outlined in the relevant protocol internally.
Proposed Reference Group:
- Director of MMP
- Strategic Advisor to the Director of Ops (SO2 lead)
- Special Envoy on Humanitarian and Development Affairs
- Head of WatHab
- Head of ECOSEC
- Head of Health
- Head of Diplomatic Strategy
- Head of Partnerships
- Head of Movement Partnerships, Policy and Legal
- Head of RM CoE Development Actors
Evaluation Quality Assurance
The main evaluation deliverables should meet the established quality standards. The quality criteria (checklists) for evaluation products (inception reports and evaluation reports) will be provided to the evaluation team in advance. Drafts of these key products will be quality assured through the ICRC’s QA mechanism which will provide feedback to the supplier via the Evaluation Office.
Publication of final report
Evaluation report and/or a summary of the evaluation (for example, the Executive Summary of the report and/or the visual communication product) will be published on the ICRC website in line with the ICRC’s Access to Information Policy.[1]
Dissemination of findings
The findings, conclusions, and recommendations will be presented by the commissioning unit internally to relevant audiences.
Follow up of recommendations
To strengthen the use of the evaluations at the ICRC, fostering ownership over the process of change and ensuring accountability for results, the intended users of this evaluation will initiate the management response process as a follow-up action facilitated by the Evaluation Office. The Evaluation Office will support the Partnership Office Team and respective stakeholders in developing and tracking the management response actions through dedicated systems and processes.
[1]https://www.icrc.org/sites/default/files/document_new/file_list/access-information-policy.pdf
Budget range or anticipated working days
The maximum budget for this evaluation is CHF 225,000, however overall cost will be taken into consideration for the selection decision. The firm will have to propose a workplan to meet the requirements and demonstrate the level of effort required for each team member at each phase of this evaluation.
Required expertise and experience
The evaluation team should possess the following combined skills and qualifications:
Team Leader:
- Substantial experience (minimum 15 years) in leading teams and designing and delivering rigorous evaluations in the international and humanitarian sectors, meeting international evaluation quality standards.
- Demonstrable track record in leading strategic and large institutional evaluations; strong understanding of approaches to the Nexus, Humanitarian Development Continuum, the Grand Bargain, sustainability, organizational maturity models and exit strategies.
- Solid experience in evaluating programs and strategies in Fragile and Conflict-Affected Countries (FCAS), and protracted crises contexts.
- Experience in conflict analysis, partnership development and stakeholder engagement, risks analysis and management.
- Advanced stakeholder communication, risk analysis, and project management skills.
- Demonstrable experience of applied ethics in evaluations.
- Minimum academic qualification: post-graduate research methods in evaluation, humanitarian response, IHL, human rights, social sciences, or other field relevant to the evaluation of the ICRC’s approach to the Nexus.
- Fluent oral and written English.
Across the team members:
- Experience of strategic evaluations and institutional assessments in the context of humanitarian organizations.
- Experience with the Nexus, Humanitarian Development Continuum, the Grand Bargain, Sustainability, organizational maturity models and exit strategies.
- Experience in conflict analysis, partnership development and stakeholder engagement, risks analysis and management.
- Knowledge, understanding, and/or experience of working with/or for the ICRC.
- Expertise in developing and tailoring qualitative and quantitative data collection and analysis tools.
- Geographic experience in MENA, Africa, Central Asia, and Latin America
- Ability to work in English, French, Spanish and Arabic, if possible, across the team members.
Deliverables
The evaluation will generate several outputs, as outlined in the graph of the phased approach. These will include an inception report, which will provide a detailed plan for the evaluation process, as well as presentation of the preliminary findings, a draft and final report.
Inception phase
One-page briefing outlining the purpose, timing, and key messages to explain the evaluation process to stakeholders. This acts as a communication tool internal within the evaluation particularly during the data collection phase. It includes the contact details of the evaluators, the evaluation commissioner, and the ICRC’s integrity weblink.
Inception report (draft and final after feedback) with:
- Preliminary findings and data gaps
- Key methodology, sampling approach, limitation and mitigating solutions
- Tools for KII, FGDs, and surveys (if relevant)
- Case study questions (if relevant) and research methodology
- Field work plan
- Matrix
The inception report should not exceed 30 pages without the annexes that would include the tools, the work plan, the case study questions and the matrix.
Data collection phase
A preliminary findings presentation at the end of the data field collection to the Evaluation Advisory Board.
The analytical and writing phase
The draft report that should not exceed 50 pages without the annexes.
The draft and final reports be composed of:
- An executive summary of 4 to 5 pages max that can serve as a standalone document
- Intro about Partnership for SHI
- Methodology, sampling, and limitations with mitigating solutions
- Findings
- Lessons learned and best practices
- Conclusions
- Recommendations
Annexes
- Terms of References
- Matrix
- List of stakeholders interviewed anonymized
- List of documents consulted
- Tools
A communication product or brief will be composed of key information- the template will be sent to the evaluation team by the evaluation office once the final report has been approved.
Please see the RFP document with Appendix 2 for full details on the requirements of the assignment and the requirements of the proposal submission: Evaluation of the ICRC’s Approach to Partnering for Sustainable Humanitarian Impact | ALNAP
How to apply
Summary of key dates
- 16 June 2023 - RFP launched with suppliers
- 30 July 2023 - Questions to the ICRC
- 07 July 2023 - Responses to Questions from the ICRC
- 28 July 2023 23.59pm - Deadline for proposals
- 14 & 15 August 2023 - Interviews for shortlisted proposals
- By end of August - Contract award
Firms are requested to submit the following:
- Maximum 10-page proposal, plus annexes, CV, Financials, Matrix and other relevant requested documents. The proposal should highlight how you would approach this evaluation, plus a proposed timeline, a workplan and the level of effort per team members. Underline the relevance of each team member to meet the evaluation’s objectives.
- CVs of consultants including descriptions of relevant assessment/evaluations or projects the consultant was involved in previously, and at least 2 referees (names and email addresses).
- At least 2 examples of previous evaluation reports for which the Team Leader was responsible for similar assignments.
- Financial proposal (including daily rates and any anticipated travel costs) – use Appendix 2 “Response Grid”
To: evaluation@icrc.org by 28 July 2023 (23:59 hours). Please indicate “Evaluation of for Sustainable Humanitarian Impact” in the subject line.
Please see the RFP document with Appendix 2 for full details on the requirements of the assignment and the requirements of the proposal submission: Evaluation of the ICRC’s Approach to Partnering for Sustainable Humanitarian Impact | ALNAP