The Real Story Behind Third-Party Monitoring in Afghanistan
MSI has implemented the MSI-Afghanistan Reconstruction Trust Fund (ARTF) project since 2016. We help the World Bank monitor the quality and impact of work on a variety of projects under its portfolio. Our team works across many sectors including agriculture, democracy and governance, education, health, irrigation, and women’s economic empowerment.
I’ve been called a lot of names in my life. One of the odder ones is “the donor’s personal police.”
That’s far from the case. So, let’s talk about what third-party monitoring or TPM really is. It can be described simply as ensuring donor programs are followed and their targets achieved.
But it’s so much more. Our role as third-party monitors covers a broad range of functions that support donors, with disparate perceptions. Donors call us their “eyes and ears on the ground,” while partners and government officers have called us both friend and foe. The truth is that TPM in Afghanistan provides all stakeholders with objective information to support data driven decision-making, while helping the World Bank build the capacity of ministry project teams to monitor communities and partners on their own.
To understand the relevance of such monitoring work, consider this article from 2018 about three Virginia men who defrauded the US Department of Defense over $9 billion of contract work in Afghanistan. They used photographs to misrepresent progress of warehouse construction and re-routed supply chains through Iran to reduce travel costs, ignoring their contractual agreement.
The fraudulent behavior reported in the article could have been identified long before it became problematic with the early introduction of TPM, but there is far more to TPM than potentially identifying fraudulent behavior.
“We are definitely not the donor’s police, I mean look at me, I couldn’t work for the police if I wanted to.”
In the past two years, I have opened more than a few meetings with Afghan ministries by saying, “We are definitely not the donor’s police, I mean look at me, I couldn’t work for the police if I wanted to.” While this line serves foremost as an icebreaker for initial meetings with ministry teams, it is absolutely true. TPM is not about looking for mistakes and reporting when things go wrong. It is about helping our Afghan partners learn to collect, analyze and use data to make more informed decisions about how and where they spend their resources.
Collecting Data in Challenging Areas
When people ask me, “how do you collect verifiable data when you can’t go to the project site?”, my answer is simple. “I don’t.” The operating environment of Afghanistan restricts my movements; I stick out like a sore thumb.
We rely on the combination of our field teams’ skill, the training we have instilled in them over many years, and the appropriate use of technology - combined with three mechanisms for primary data collection - to comprehensively cover World Bank funded projects. The mechanisms utilize local field engineers, female enumerators and citizen monitors who can safely access areas of the country where foreigners can’t go.
In January of 2018, our role in Afghanistan evolved from largely infrastructure monitoring to include process monitoring and beneficiary impacts. We trained field engineers to double as enumerators for projects requiring interviews with male community members. The engineers learned how to recognize and mitigate causes of researcher bias such as confirmation and leading question/wording bias. After a year of ongoing (bi-monthly) trainings, our field engineers have proven to be efficient interviewers of male community members in addition to their primary responsibilities inspecting the structural integrity of infrastructure projects.
However, accessing Afghanistan’s women was perhaps the biggest challenge in our evolving role. Attitudes toward women vary substantially by and within provinces, and it is not possible (nor culturally appropriate) for men to conduct interviews with female community members. To overcome this challenge, we hired 22 women from various provinces and trained them in the same social science methods as our field engineers.
As a result, in the past year they were able to obtain valuable information about female inclusion in decision making processes and perceptions of the value and impact of World Bank projects. In fact, our female team members provide some of the most useful information collected by our team.
Our Citizen Monitor Approach
To verify projects under construction in remote or so-called Armed Opposition Group-occupied territories, we established a “Citizen Monitor” program for road and canal construction projects to support our field engineers and female enumerators.
Here’s how it works: local community members are selected by MSI field engineers from remote or volatile communities to provide photo observations of ARTF-funded sub-projects. A sub-project can be a road, a bore well, solar mini-grid, even a women’s economic empowerment self-help groups or village savings and loans associations. Our field engineers and female enumerators visit a site one time, but citizen monitors provide ongoing photographic evidence, using a free mobile application. They take geo-tagged and time-stamped photos which are then uploaded and reviewed by quality assurance engineers in our Kabul office on a rolling basis. We can identify deviations from the design drawings and monitor construction progress. As a result of its initial success, the Citizen Monitor program was introduced to two additional ministries to begin similar monitoring.
Using a combination of citizen monitors, female enumerators, and field engineers, we’re able to customize our approach to the specific needs for TPM on a given project. For instance, for projects requiring predominantly construction monitoring, we use field engineers for the initial site visit of ongoing construction and include citizen monitors for continuous monitoring to identify deviations and report progress.
Where beneficiary perceptions and process monitoring are combined with construction monitoring, we combine field engineers with female enumerators to get the complete story from male and female beneficiaries in the community. The mixed gender teams produce exceptional results given the cultural sensitivity around women’s participation in community development projects.
Finally, we have teams of multiple female enumerators, each traveling with a male family member (Mahram) to monitor projects designed to improve the economic opportunities available to women.
All teams provide photographic evidence with time stamps and geo tags to ensure that our findings are accurate and valid. We use technology that is free and easy to use to do so. The simple solutions we use to verify construction progress, monitoring team locations and evidence of process compliance, are all ways of preventing the kind of fraud illustrated in the article I linked earlier.
It’s all well and good that we provide these kinds of data to the World Bank but the real value is in passing our knowledge to the Afghan ministries. Specifically, I believe our greatest value comes from sharing every step of our approach and helping our government colleagues learn how to manage the design and implementation of monitoring systems into the future.
Our World Bank colleagues are very supportive of how we build monitoring capacity within ministries. They appreciate that our field engineers are accompanied by ministry engineers to most construction sites, where we train ministry teams to use the same free applications to collect verifiable data. Although there are always disagreements between engineers about the severity of a construction deviation, we all agree that the systems used to record and transmit verifiable data are of value to everyone.
Assisting ministries in designing instruments for data collection, implementing modern technology to modernize ministry teams’ approach to data management, and providing the World Bank with actionable data make it easy to see that TPM is far more than the being the donor’s personal police.