Methodology Flow Chart
This index came about because many people—defence leaders, defence officials and members of civil society—repeatedly asked us: ‘how can we measure our progress in combating corruption in this sector?’ This index assesses and compares levels of corruption risk and vulnerability across countries to increase understanding of the problem and encourage reform. The methodology ensures that we can compare results over time and between countries to monitor progress and encourage change.
Each country is placed in a band from A to F. These bands are based on scores on an assessment consisting of 77 questions—for each question, the government was scored from 0-4. The percentage of marks overall determined which band the government was placed in. Countries were also scored in five risk areas: Political risk, Financial risk, Personnel risk, Operations risk, and Personnel risk.
We considered a range of institutions in each country: the defence and security ministries and armed forces in each country, of course, but also any other government institutions with the potential to influence levels of corruption risk in the sector.
Researchers assessed each government individually, using a questionnaire of 77 questions based on our defence corruption risks typology. Each country assessment was peer reviewed twice. We also invited the governments to review the drafts. Where governments responded late, we invited them to submit a report in response instead. Lastly we invited TI National Chapters to undertake a review.
Ultimate ownership and responsibility for the finalised scores and bands rests with Transparency International UK's Defence and Security Programme (TI-DSP).
For a comprehensive explanation of the methodology, please download the full methodology document, a shorter version of our methodology, a list of the 82 countries assessed, the questionnaire with model answers and without them, and the dataset.
Each government was assessed using a questionnaire of 77 questions. Each question is scored from 0 to 4. Model answers are provided to guide scores. The score and model answer for a score of 4 indicates best practice for each question.
For each question the assessor provided a score, a narrative explanation for the score, and a list of sources used.
This questionnaire is based on our typology of defence corruption, which defines five main areas of corruption risk:
- Political risk
- Financial risk
- Personnel risk
- Operations risk
- Procurement risk.
These five categories are further broken down into 29 sub-categories of corruption risk.
The questionnaire was developed by Transparency International UK's Defence and Security Programme, with the help of a team of experts with practical experience in the defence and security sector, and our academic advisory committee.
SOURCES OF INFORMATION
Assessors used a wide range of sources for each country. These sources could include government documents, websites, and legislation; academic sources; news media articles; and interviews with government officials or other experts. We have tried to find a range of sources for each question, and not to rely too heavily on a particular source. When assessors conducted interviews, interviewees remained anonymous by default to protect their privacy.
Given the level of secrecy in the defence and security sector, it was no surprise that insufficient information was a problem in several countries. Where information on a particular question was unavailable, we asked assessors to choose a score based on a reasoned assumption. Assessors were asked to consider the wider social and political context of the country, and any broader corruption problems. For example, if there was no information available about the procurement cycle in general, it is unlikely that there will be adequate transparency with regard to complex offset contracts or financing packages.
Each country’s assessment was conducted by a lead country assessor. Their research was reviewed by two independent peer reviewers.
In the next stage, we invited governments to review the assessor’s responses during the research process. Where governments responded late, we invited them to submit a report in response instead. Those reports are published on their country scorecard.
We also invited Transparency International's National Chapters to conduct a review, in those countries where Transparency International has a chapter.
Peer reviews, government reviews, TI national chapter reviews and the core team’s own reviews were used to standardise and balance assessors’ scores. This created a dialogue between assessor and their reviewers, which can be read in each country’s assessment. You can watch a video to understand the research process from an assessor’s position.
The core research team was drawn from academia, journalism, and the anti-corruption movement, including our own regional TI chapters. Researchers were recruited on the basis of their expertise and independence, and as much as possible we sought individuals in-country, who had access to knowledge on the ground.
The team at Transparency International UK's Defence and Security Programme (TI-DSP) carried out standardisation and finalisation processes, and scores are our responsibility. Research was carried out between July 2011 and October 2012.
A total of 82 governments were assessed in this Index. We aimed for a diverse range of countries, including major arms importers and exporters, those in conflict or post-conflict, and countries from all regions of the world.
We used six objective criteria to select the countries:
- Aggregate arms exports volumes, 2001-2010 (data from SIPRI http://www.sipri.org/databases/armstransfers)
- Aggregate arms imports volumes, 2001-2010 (data from SIPRI http://www.sipri.org/databases/armstransfers)
- Size of military (absolute terms), 2010. (data from the Military Balance, from the IISS http://www.iiss.org/publications/military-balance/ )
- Size of military (per capita), 2010. (data from the Military Balance, from the IISS http://www.iiss.org/publications/military-balance/ )
- Size of police force (absolute terms), 2003-2008. (data from United Nations Office on Drugs and Crime. http://www.unodc.org/unodc/en/data-and-analysis/statistics/data.html. The average yearly figure from available data in the period 2003-2008 was used.)
- Size of police force (per capita), 2003-2008. (data from United Nations Office on Drugs and Crime. http://www.unodc.org/unodc/en/data-and-analysis/statistics/data.html. The average yearly figure from available data in the period 2003-2008 was used.
Because criteria 5 and 6 relate to the police specifically, and not the internal security services, they were given a weighting of fifty per cent.
Three subjective criteria were also used. First, we sought to ensure geographic diversity so there was a wide range of countries and regions. We also selected countries that were likely to have enough information available to be analysed. Finally, we chose countries that were likely to display substantive findings.
Governments were assessed using a questionnaire of 77 questions. For each question, a score from 0 to 4 was awarded.
This table describes the meaning of each score:
QUESTION SCORING PRINCIPLES:
|4 = High transparency; strong, institutionalised activity to address corruption risks.|
|3 = Generally high transparency; activity to address corruption risks, but with shortcomings.|
|2 = Moderate transparency; activity to address corruption risk with significant shortcomings.|
|1 = Generally low transparency; weak activity to address corruption risk.|
|0 = Low transparency; very weak or no activity to address corruption risk.|
Based on the overall percentage score, each government was placed in a band from A to F, using the following scheme:
|BAND||LOWER SCORE (%)||HIGHER SCORE (%)||
Risk area scores: Scores were also broken down into five key risk areas: political risk, financial risk, personnel risk, operations risk, and procurement risk. The questionnaire was split into these five categories, and the risk area scores were calculated using the percentage of marks awarded for the questions pertaining to each risk area.