Evaluation of the Detention Programmes and Activities by the ICRC Panama Delegation

1. Background information and rationale of the evaluation

The ICRC Panama delegation is seeking to evaluate the detention activities it has been implementing within the Panama prison system from 2016 until today.

The work in favour of detainees is a central feature of the ICRC. The organization visit people detained in relation to armed conflicts, for which it has a specific monitoring mandate under international humanitarian law. It also acts on behalf of people deprived of their liberty in other contexts characterized by violence, social tension and unrest.

ICRC’s priority is to ensure that detainees are treating humanely and with respect of their dignity, regardless of the reason of their detention. Its work in detention is rooted in visits to places of detention and is resolutely beneficiary-centric. It is also marked by its multidisciplinary approach, the aim of which is to effectively analyze and address the humanitarian concerns faced by people deprived of their freedom.

1.1 The ICRC's work in detention in Panama

ICRC started its detention action in 1989, visiting prisoners of war in Panama following the incursion of the US Army. ICRC then focused its work on persons held in relation to the Non-International Armed Conflict (NIAC) in Colombia (1990s and 2000s). Early 2015, the delegation started developing a system-based approach, with three main areas of work: prison maintenance, access to health and hygiene. This approach was framed within a strategy document spanning till 2019.

Within this framework, a special attention was given to a project Ecosolidos initiated by persons deprived of liberty (PDL), aiming at improving the prison environment by recycling large amount of rubbish in one prison. Two other projects were subsequently added. The Sembrando Paz project saw the creation in various prisons of vegetable gardens and tree nurseries (which are later planted in the community). The maintenance projects were put in place to train people deprived of liberty so they could acquire new skills in maintenance whilst improving prison conditions.

Those projects are now the backbone of the current ICRC action, which mainly supports the authorities and the PDL in running and implementing those projects.

In parallel to this specific action, the ICRC is currently developing a framework of construction criteria for prisons (CETIP[1]) with authorities of 10 Latin American countries. Panama authorities will remain an active member of this process.

1.2 Key data on the prison system in Panama

The General Directorate of Penitentiary Systems (DGSP) is an agency of the Ministry of Interior[2], responsible for the correctional services in Panama.

As of August 2022, 21,113 persons are deprived of liberty in Panama, of which 65% are convicted and 35% on remand. Panama has 19 penitentiary facilities, including Las Joyas complex which houses around 59% of the total prison population in Panama; this is where ICRC is mainly implementing activities.

1. Aims, objectives and scope of the evaluation

2.1. Aims and objectives

The purpose of the evaluation is to review the current approach to detention action in Panama, in order to understand its relevance (and appropriateness), effectiveness, and how the activities contribute to sustained results. The evaluation aims to critically assess the current approach in its context and produce findings and recommendations which will inform ICRC strategic thinking and engagement at regional and global levels in the field of detention.

The specific objectives of the evaluation are to:

• Analyse and assess the relevance and appropriateness of the delegation’s approach to detention, in relation to the needs of persons detained, the context including the role of others (authorities, international organisations, NGOs etc), and the ICRC’s global detention strategy as well as the detention action framework.
• Analyse and assess the effectiveness of the delegation’s approach to detention i.e., are the activities meeting the intended objectives?
• Analyse and assess the extent to which the approach to detention makes a difference to persons deprived of their liberty, and whether any changes or results continued (or are likely to continue) in the longer term.

The evaluation draws on the OECD DAC criteria of relevance (appropriateness), effectiveness, and impact combined with sustainability. Elements of the criterion of efficiency (use of resources) are incorporated into effectiveness rather than a distinct criterion.

The results of the evaluation are meant to be used internally, the primary audience of this evaluation being the ICRC, in particular its Detention Unit in Geneva and the Panama Delegation.

2.2. Scope

The evaluation will review the approach to detention holistically, incorporating all activities, initiatives, and projects.

Timeframe to be assessed: The detention activities implemented in between 2016-2022.

Key evaluation criteria and guiding questions: the ICRC’s detention programmes should be evaluated within the backdrop of recognised legal and humanitarian standards, including the ICRC’s own institutional frameworks, international human rights law (IHRL), other international standards such as the 2015 UN Standard Minimum Rules for the Treatment of Prisoners (the Mandela rules)[3] and Panamanian legislation governing the deprivation of liberty.

As set out in the aim of this evaluation, important criteria have been identified below as relevant for this evaluation process. Some questions under the defined criteria have been suggested to be refined with the evaluation consultants during inception phase. The list is not exhaustive.

2.3. Evaluation key questions

Criteria: Relevance and appropriateness of ICRC action

The purpose of using the criteria of relevance and appropriateness is to analyse whether the approach is ‘doing the right thing’.

1. How aligned is the approach to detention (strategy and action) with the needs of persons detained? How were these needs defined and how does the approach to detention ensure it remains relevant?
2. How aligned is the approach to detention with the ICRC’s detention action framework, its institutional detention strategy, and other relevant institutional documents?
3. To what extent do the authorities share the ICRC’s analysis of its key concerns in detention and to what extent are the authorities willing to support and work with the ICRC in achieving shared objectives?
4. How coherent are the aims and objectives of the approach and strategy for detention in the delegation: are they well-defined, aligned with needs, achievable, and measurable?

Criteria: Effectiveness

This criterion examines whether the intervention is achieving its stated objectives. Some elements of efficiency (use of resources) are included here.

1. How comprehensively and holistically are the activities within the approach to detention being implemented? Are the needs of detainees addressed through an effective multidisciplinary approach?
2. How does the implementation of activities result (or will likely result) in the achievement of objectives?
3. What are the internal and external enabling factors and barriers to implementing activities and/or achieving objectives?
4. Are human resources adequate to implement and fulfil the objectives of the detention strategy (both human resources for the detention team and the wider disciplinary working with colleagues)?

Criteria: Impact and sustainability

This criterion examines whether the intervention is making a difference, and whether the benefits (changes or results) last or will last.

1. What differences (changes or results) has the approach to detention made, in what ways, and for whom? What anticipated changes or results were not achieved, and why?
2. Which changes or results have lasted, or are likely to last?
3. Has there been any unintended results, either positive or negative?

Additional component: Lessons Learned

1. What lessons learned could be applied to other contexts for the ICRC’s detention approach?
2. Approach and Methodology

3.1 Mixed qualitative methods

A criteria-based assessment is anticipated consisting primarily of mixed qualitative methods such as key informant interviews (primary data collection) and document reviews (secondary data sources). It will need to consider the specificities of the country and context such as specific legal frameworks and structure of the governmental institutions.

Below are some examples of data collection tools that the ICRC would consider pertinent to this evaluation:

• Desk review of all key documents.
• Semi-structured interviews with previous and current staff members (where possible).
• Semi-structured interviews with stakeholders from the prison services. Interlocutors will include those working at central prison authorities’ level and prison level.
• Semi-structured interviews with PDLs and ex-PDLs – the ICRC will facilitate the organisation of the prison visits. Instructions and guidance (including ethical protocols) regarding this aspect will be provided in due time.
• Semi-structured interviews with other relevant stakeholders as well as families of PDLs.

Interviews with authorities and any other relevant stakeholders where possible should be done in-person and may happen virtually if there is no other option (the ICRC can facilitate this). The suggested list of stakeholders and minimum list of documents to be reviewed have been provided in Annex 1. The list will be refined during the inception phase based on discussions between the detention team, Panama Delegation, and the consultants.

The final methodology to be adopted for this evaluation will be defined by the consultants’ proposal and the objectives of this evaluation. It will be formalized in an inception report that will provide the detailed methodology, sampling strategy, workplan, research matrix, and include the list of documents reviewed for this initial phase. The inception report will be validated by an internal evaluation ‘steering committee’.

However, it is intended that the selected consultants will devise and set out in the inception report a predominantly qualitative and participatory approach. The inception report should also consider methods for dissemination of findings to stakeholders, taking into consideration the confidential nature of the ICRC’s work and the different stakeholder groups involved.

The delegation will be available to facilitate access for the consultants to the necessary information as well as facilitate introductions and communications with the relevant stakeholders.

3.2 Potential risks and limitations

• Availability of delegations and detention team members to engage with the evaluation team
• Authorities’ reluctance to cooperate on the evaluation, share certain data, meet with the evaluators and/or provide them with access to the places of detention where needed.
• Access to the PDLs and ex-PDLs and their willingness to participate.
• Complexity of understanding and evaluating detention within a complex legal framework, authorities, and monitoring bodies, in the allocated time.

The consultants should elaborate any further risks and limitations and propose mitigation measures in the inception report.

1. Ethical considerations

The consultants must declare and guarantee that they Commit to Professional Standards for Protection Work, the ICRC’s Code of Ethics for Procurement, the ICRC Rules on Personal Data Protection and the ICRC code of conduct. These include:

• Non-discrimination (on the basis of sex, disability, race, religion or belief, political opinion, sexual orientation, national origin, age, class, language, or any other characteristic) and impartiality (the evaluation should be based on the needs of the persons deprived of liberty and not on any other ground).
• Do no harm.
• Respect for the dignity of affected persons.
• Securing informed consent of all interlocutors and maintaining their anonymity
• Confidentiality: the detention file is sensitive and requires a high level of confidentiality during and after the collection of the information. The communication during and after the evaluation process should abide by the ICRC rules in this regard.
• Protection of the data collected.

4.1 Ethical review process

The decision to engage with PDLs and ex-PDLs during the primary data collection phase will be made during the inception phase in consultation with the evaluators. At the proposal stage, the evaluators are asked to explain the rational for the evaluation design, and if engagement with the PDLs and/or ex-PDLs is needed based on the objectives and scope of the evaluation. The evaluation design will require formal ethical review board approval if primary data collection will include PDLs and ex-PDLs. The ICRC will manage the process of submitting the inception report for the ethical review board approval and the evaluators need to consider that in the timeline for the assignment.

1. Evaluation management

The evaluation will be managed by the commissioning delegation through an internal Evaluation Manager. The Evaluation Manager will be the day-to-day contact for the external evaluation team and will oversee the whole process.

The Evaluation Advisory Group will accompany the process and provide expert advice and feedback at key stages. It comprises (at minimum) of a Detention Adviser, the Head of the Evaluation Unit as well as the Heads of Sector for the relevant region (health care in detention, protection and water and habitat). This engagement is particularly important but not limited to the main products of the evaluation i.e., the TOR, draft inception report, and draft evaluation report. The membership of the advisory group is internal to the ICRC. The protocol for the advisory group is set out in its dedicated ToR.

The Evaluation Unit will be engaged in the process by providing technical advice on the process and the main evaluation deliverables directly and through the established Evaluation Quality Assurance system. The quality criteria (checklists) for evaluation products (inception reports and evaluation reports) will be provided to the evaluation team. Drafts of these key products will be quality assured through the ICRC’s QA mechanism which will provide feedback to the supplier via the Evaluation Office.

1. Responsibilities and required skills

The assignment is expected to be undertaken by a team of consultants who possesses a complementary skillset in evaluation, detention expertise including good knowledge of the ICRC’s work in detention.

As the delegation has limited HR resources working on its detention programmes and activities, potential support and accompaniment to the consultants may be limited and the consultants should have the ability to work independently with guidance.

The consultants will:

• Assume full technical responsibility for the services provided for the development of this consultancy in accordance with the terms of reference in this document.
• Guarantee production of high-quality reports and services, being attentive of delivery times, writing and presentation of reports with technical standards, until the final approval of the deliverables.
• Maintain consistent communication and coordination via e-mail, telephone, or face-to-face interviews with the ICRC Evaluation Manager, so that they can report on progress, limitations, difficulties, and recommendations.

The consultants should have the following qualifications and competencies:

Evaluation expertise

• Proven record in leading evaluations in the protection or detention fields. Experience of leading and/or participating to evaluation in a field related to prison, criminal justice or social activities is a strong asset.
• Knowledge of and experience in applying standard evaluation principles, and qualitative evaluation methods.
• Ability to draft concise evaluation reports of high quality in English and Spanish**.**
• Ability to work in a team.
• Self-employed status (must provide proof in country of origin) or company registration status.
• Availability to travel to for inception and/or data collection activities during the evaluation period (probably early 2023).

Detention expertise

• Substantial professional experience in detention and/or criminal justice related fields.
• ICRC experience in detention work and prison systems is a strong asset.
• Ability to draft concise evaluation reports of high quality in English and Spanish.
• Strong analytical skills.
• Ability to work in a team.
• Availability to travel for inception and/or data collection activities during the evaluation period (probably early 2023).

1. Deliverables

The evaluation will include the following deliverables in English and the ICRC will make the translation of the final products available in Spanish, if deemed necessary.

• Inception report: the consultants shall prepare an inception report prior to the fully-fledged data collection exercise. Max 20 pages.

  • The inception report shall present a structured evaluation plan containing the methodology and an Evaluation Matrix showing how the evaluation questions will be answer (including sub-questions, indicators or judgement criteria, data sources and data collection methods) and data collection tools.
  • The first draft inception report should be submitted and presented to the Evaluation Manager for validation within three weeks after the start of the process.
  • The Inception Report will meet the criteria set out in the ICRC quality assurance checklist for Inception Reports.

• 1 page brief that introduces the Evaluation to interlocutors

• Validation meeting: prior to the submission of the first draft of the evaluation report, the consultants should present the key findings of the evaluation to the Evaluation Manager and Evaluation Advisory Group, plus selected ICRC colleagues. This provides an opportunity for the ICRC to provide any initial feedback on the key findings and discuss the scope and suitability of recommendations.

• Final evaluation report to be submitted to the Evaluation Manager. Max 40 pages excluding annexes

  • The consultants must produce one report presenting the findings, conclusions, and recommendations, based on evidence and responding to the objectives and questions of the evaluation.
  • The report should be as concise as possible. Annexes to the main text are possible.
  • The Evaluation Report will meet the criteria set out in the ICRC quality assurance checklist for Evaluation Reports.
  • Possible structure of the evaluation report: executive summary; introduction/background of the evaluation; description of methodologies used, limitations and mitigations strategies, sample strategy; findings on the delegation’s detention programmes and activities; lessons learnt; recommendations and a conclusion.

• Presentation of key findings: following the submission of the first draft of the evaluation report, the consultants should present the evaluation to the Delegation and selected ICRC colleagues in Geneva (one presentation).

• Briefing paper on the Recommendations and their rationale. Max 5 pages.

All deliverables will remain as the intellectual property of the ICRC for an indefinite period. Both the inception report and evaluation report will adhere to the ICRC’s evaluation quality standards, to be provided to the consultants at the start of the assignment. The briefing paper and/or final report will be published on the ICRC website.

1. Timeline of consultancy

The evaluation is anticipated to require a maximum of 90 days of paid consulting days over 4-5 months. The ICRC is flexible with regards to start date and does not mandate that working days must be consecutive. However, the ICRC envisages that the consultancy will mainly be done virtually/remotely, with a maximum of 3 weeks country visit.

• The evaluation will commence as soon as possible after the appointment of the consultants.
• The deadline for the finalization of the evaluation process and submission of the final report of the consultancy to be determined

Indicative timeline

Activity/Deliverables - Time required

Inception phase - 2-3 weeks

Data collection phase - 3-4 weeks (remotely) and/or 5-6 weeks (country visit)

Draft report and preliminary presentation - 3-4 weeks

Final report and briefing paper - 2-3 weeks

1. Budget

40’000 CHF max including consulting days and travel if required

1. Other

Any travel and accommodation required must conform to the ICRC travel policy and parameters, including maximum allowances, with arrangements confirmed at contracting stage.

[1] CETIP stands for Criteria for Technical Standards in Prison Infrastructure (Criterios de Estándares Técnicos en Infraestructura Penitenciaria).

[2]https://www.sistemapenitenciario.gob.pa/historia/

[3]A/RES/70/175 (undocs.org)

How to apply

Please send:

• Maximum 2-page proposal highlighting how you would approach this evaluation including a suggested proposed timeline/workplan
• CVs of consultants
• List of relevant assessment/evaluations or projects you were involved in
• Sample evaluation report of a similar assignment
• Financial proposal (including daily rates and any anticipated travel costs).

To: pan_hr_services@icrc.org by 14 November 2022. Please indicate “Detention Evaluation” in the subject line.