As people around the world look to support earthquake relief efforts in Nepal, scholars from Stanford and the London School of Economics and Political Science offer new research that can help donors make better decisions about where and how to contribute their money.

“NGO reports tend to focus on quantity in delivery, such as numbers of homes and people served—but not on quality,” write Yong Suk Lee (Stanford) and J. Vernon Henderson (LSE).

In a forthcoming paper, the coauthors evaluate reconstruction efforts in Indonesia following the 2004 earthquake and tsunami, and find two trends: aid agencies that directly execute their services—point-to-point—perform the highest quality work. And, when agencies contract their services, higher quality work is performed when a global, not domestic, implementer completes the work.

Knowing this reality, and with improved disclosure of outcomes, the coauthors hope that donors would be able to make more informed choices.

Fishing village survey 

Figure 1. A map details the survival rate of the population and flood damage within northern Indonesia in 2004. Darker shaded areas show a higher survival rate, lighter shaded areas show a lower survival rate. Striped areas denote flooding, largely on the northeastern border. Boundaries marked with thicker lines are ‘kabupaten,’ or county divisions, and lightly colored lines are ‘kecamatan,’ or sub-county units larger than a village alone. (Courtesy of Yong Lee).

Through fieldwork and three rounds of surveys – in 2005, 2007 and 2009 – Henderson and Lee investigated aid work in Aceh, an area of coastal villages in northern Indonesia (Figure 1).

Humanitarian efforts there focused on “hard aid” such as construction of houses and fishing boats. Total aid delivered amounted to $7.7 billion and was implemented by international and domestic aid agencies—some directly and some as contractors—as well as the Indonesian government.

First, Henderson and Lee conducted a pilot survey, and then with a cohort of surveyors from the University of Indonesia, held interviews with village leaders and fishing families. Participants were asked to rate their housing accommodation, and if applicable, how their fishing activity compared to before the disaster.

“Mostly, we sat with villagers to see how willing they were to talk about aspects of aid,” Lee said. “Since it was several years after the tsunami hit, people were pretty open throughout the process.”

Data from those surveys was combined with information from the Recovery Aceh-Nias relief project database maintained by the government and the U.N., as well as demographic information provided by participants.

Delivering aid: Global v. local

Empirical analysis revealed that aid agencies such as the Red Cross and Catholic Relief Services reflected higher quality aid delivery (at a mean quality near 3.00), while agencies such as Save the Children and Concern Worldwide reflected lower quality (at a mean quality between 1.0-1.5).

“What’s surprising is that reputation didn’t really line up with what was expected,” Lee said, citing a few renowned agencies that didn’t receive high marks.

Lee said this could be explained by the fact that aid agencies that specialize in disaster recovery are better equipped, while a learning curve might exist for agencies with wider missions.

Global aid agencies are more likely to have logistical experience given their reach across multiple disaster situations. And while all NGOs face reputational costs for their results, global aid agencies are greater exposed to criticism because, by size, they’re more visible.

Yet, while global aid agencies and implementers may have the raw skills, local implementers have the cultural know-how.

“Local implementers might not have the most experience – like how to construct a house or manufacture a fishing boat – but they will likely know what’s actually desired,” Lee said. “So, there are obvious tradeoffs at play.”

For example, villagers reported bad ventilation in houses. This was because some aid agencies used small windows and concrete instead of wood material more traditionally used in Indonesia. Some boats were impossible to use because of improper design; they sank upon first use or fell apart after a few months.

Image

Collection of photos from fieldwork in Aceh, Indonesia, provided courtesy of Yong Lee. Upper left: A house built in an aid project village shows windows retrofitted after initial construction. Upper right: Boats constructed by aid agencies for fishing activity are refashioned to serve as water taxis for people and cars. Lower: Fishing boats sit unused on the side of the road many of them impossible to use, according to villagers surveyed.

Logistics and oversight

Aid delivery depends in many ways on the location and scale of the disaster. But, a few main aspects can determine if an aid agency doing its own work or operating as an implementer meets or exceeds expectations.

Henderson and Lee suggest that agencies that were highly supervisory had greater positive outcomes from their workers. In the case of Aceh, better monitoring and insistence on quality by leadership is a likely corollary between construction of better quality homes and boats.

“Rather than just give money, NGOs need to really oversee the projects. Organization and management are essential facets,” Lee said. “And that requires a lot of additional effort on their part.”

Oversight is especially relevant in disaster situations because of the often-overwhelming need for reconstruction. A flood of less-skilled workers enters the market to fill this gap, and on average the quality of work degrades.

“It’s much more difficult to impose quality control at this point,” Lee said. “So the implication that comes out of it is how does the implementer effectively utilize less-skilled workers.”

Getting to know the implementers and evaluating their work in-progress would help ensure quality on behalf of the aid agency. And, better dissemination of information about aid outcomes would help assure donors that their monies are being applied in the best possible way.

Future study

Most “hard aid” delivered to Aceh’s villages had finished by 2010, but “soft aid” such as democracy promotion and women’s empowerment stayed longer.

Henderson and Lee conducted one final survey in 2011. The data has been offered as open source material for researchers along with the larger data set.

Noting this, Lee said, “We’re thrilled that people are looking into the data further. It’s exactly what we wanted.”

Research projects applying the data include the impact of the tsunami on Aceh’s local economies and health effects on the population, among other areas.

Stanford scholars are urging Japan to take advantage of an upcoming opportunity to show clear, heartfelt remorse for its actions surrounding World War II.

Making such amends will give Japan credibility as it seeks to assume a global leadership role well into the future, they say.

On Aug. 15, Prime Minister Shinzo Abe will publish a short statement to commemorate the 70th anniversary of the end of World War II, which follows similar practices of his predecessors.

Stanford's Walter H. Shorenstein Asia-Pacific Research Center (APARC), which has long advocated wartime reconciliation in Asia, recently issued a 15-page report in English and Japanese featuring eight hypothetical statements suggesting what the Japanese prime minister might say in his August address. The report, which is available in both English and Japanese, was recently made available to academics, media and the general public and has already received interest from the Japanese media.

The wording of Abe's statement will be scrutinized by governments and experts in Asia and around the world, the Stanford scholars say. During WWII, China and Korea, as well as other Asian nations, endured brutal Japanese military occupations.

"Many have been speculating what the (Abe) statement will be like," wrote Takeo Hoshi, director of the Japan Program at the Shorenstein Center, and APARC associate director for research Daniel Sneider in the report.

For example, Hoshi and Sneider asked, will Abe follow the direction set by prior Japanese prime ministers by expressing remorse for the suffering of Japan's Asian neighbors while apologizing for past aggression and colonization? Future collaboration in world affairs is also important, they added.

"We asked our colleagues what they would say in the 70th anniversary statement if they were the prime minister of Japan, and to write their own version of the statement," Hoshi and Sneider wrote.

"Our goal is to understand the diversity of reasonable views on the issue of Japan's responsibility for the cruel and violent war and Japan's role in building a peaceful and prosperous world," Hoshi and Sneider said.

The Stanford experts who wrote the statements included Hoshi and Sneider as well as Alberto Diaz-Cayeros, a senior fellow at the Freeman Spogli Institute for International Studies; Peter Duus, a professor emeritus of Japanese history; Thomas Fingar, a distinguished fellow at the Freeman Spogli Institute; David Holloway, a professor of international history and of political science; Yong Suk Lee, the SK Center Fellow at the Freeman Spogli Institute; and Harry Rowen, a professor emeritus of public policy and management.

For example, Fingar said in his version, "Let us also resolve to make the 80th anniversary of World War II the 10th anniversary of a more cooperative, more inclusive, and more secure region," and Hoshi wrote in his version, "To avoid any potential misunderstandings, Japan needs to recall past failures, remember the suffering of neighboring Asian peoples, and reaffirm the commitment to world peace more than ever."

On the subject of women, Lee's version noted, "The war and Japan's colonial rule created much suffering, but I would like to especially ask forgiveness to the women from many nations who suffered under colonial rule."

In August 2014, the Shorenstein Asia-Pacific Research Center issued a report on a Stanford-hosted dialogue on World War II memories in northeast Asia. Heightened tensions the last few years among the governments of China, Japan and South Korea have revolved around territorial disputes and the way WWII is portrayed in speeches and educational materials.

"Each nation in northeast Asia and even the U.S. has selective or divided memories of the past, and does not really understand the views of the other side," said Stanford's Gi-Wook Shin, director of the Shorenstein center, in a 2014 Stanford news release.

Clifton Parker is a writer for the Stanford News Service.

Responses to the project

Toyo Keizai, a leading Japanese business weekly, published all eight verisons in English and Japanese stating, "we hope this will provide an opportunity to bring about a wide range of discussion."

University of Tokyo professor Tetsuji Okazaki wrote about the project in the Asahi Shimbun (the article is in Japanese and also attached as a PDF below).