Earlier this year, I joined a team of development researchers through AidData's Summer Fellows program. In an effort to assess the state of disaster resilience in the Philippines, we embarked on a mission to (1) find all the aid funds that flowed to the Philippines in response to Typhoon Haiyan, and (2) analyze the allocation of Haiyan aid in relation to disaster infrastructure in the Philippines.
 
Why focus on this particular typhoon? Well, this category 5 tropical cyclone was responsible for destroying more than 400 thousand homes and taking over 6 thousand lives, making it the deadliest typhoon in Philippine history.
 
Based on what we found, 1.9 billion dollars were promised but only 650 million dollars were delivered. This looks bad but it is not telling you the whole story. A serious limitation that we face when studying aid is that we can only examine what is reported and made publicly available. All the aid that goes unreported or is kept private is not reflected in these findings.
​

Fortunately for us, there are several open data initiatives working to make international aid as transparent as possible. We obtained the aid data for this study from two open aid databases: (1) the Government of the Philippines’ Foreign Aid Transparency Hub (FAiTH), and (2) the International Aid Transparency Initiative (IATI).

​Aside from learning how much aid was delivered, we were also interested in learning where in the Philippines the aid funds went: what region, province, city, barangay, and so on. This proved difficult because out of the 483 aid projects that we found, only 79 contained information at the subnational level. This accounts for only 14% of the total funds disbursed.
​

The fellows and I geocoded those 79 projects and made several maps. The first map shows the number of aid project locations by province and the second map is looking at the total funds disbursed by province.
​

After looking at these maps, you might be alarmed to see so many provinces in white. According to the data that we found, only 12 out of 82 provinces received humanitarian assistance in the aftermath of Typhoon Haiyan.
 
This could be a consequence of the lack of data with subnational information. As I mentioned before, only 79 out of the 483 projects that we found contained sufficient information to be assigned a precision code below 6 or 8 - that is, the project descriptions included location information more specific than “the Philippines”. Maybe if development actors included more details about their projects when reporting their aid, these maps would look a lot more colorful.

However, if we isolate the top ten flood risk and storm surge provinces in the Philippines, we end up with a pretty similar map. This offers an alternative explanation for why so much of the country is white. Rather than being neglected by the humanitarian response, it is possible that the provinces receiving zero funds were simply not among the most affected by Typhoon Haiyan.
​

Now let’s look at the state of disaster infrastructure in these ten high risk provinces. There are three types of infrastructure that are critical in a disaster situation: health facilities, evacuation centers, and roads. Our fellow summer fellow, Prabesh Basnet, did an in-depth analysis of the road infrastructure in the Philippines in the context of disaster resilience, leaving me to focus on health facilities and evacuation centers.
 
From this side-by-side comparison of evacuation infrastructure and aid disbursements in the top ten flood risk and storm surge provinces, there doesn’t seem to be a correlation between the amount of funding received and the amount of evacuation centers in each province. However, we could take this analysis a step further if we had a time frame for when these evacuation centers where built. With that additional information, we might be able to determine whether the funding received in response to Typhoon Haiyan had any impact on the evacuation infrastructure in these provinces. 
​

​
We observe a similar lack of correlation between the amount of funding received and the amount of health facilities in each province. We could also further this analysis if we had information about when these health facilities were built.
​

​When it comes to health infrastructure, however, we face an additional obstacle: the health infrastructure map only captures those health facilities that are mapped on OpenStreetMap (OSM).
 
Why is this a problem? We ran a small experiment comparing how many health facilities exist versus how many are mapped on OSM and Google. Our findings did not fare well for OSM.
 
Of all the hospitals that exist in the top ten flood risk and storm surge provinces, 42.67% are mapped on OSM, as opposed to 89.22% on Google Maps. That means that 57.33% of the hospitals that exist in these provinces are not included in the health infrastructure map above.
 
We ran the same experiment for evacuation centers in these provinces and found that only 8.68% of all evacuation centers in these provinces are mapped on OSM, as opposed to 84.5% on Google Maps.
 
Why did we use OSM if Google Maps does a better job of mapping disaster infrastructure? The key advantage of OSM is that it is free and open source, whereas Google Maps is not. This means that anyone with a computer and access to the internet can contribute and download data from OSM.
 
That is precisely why the Map the Philippines initiative strongly advocates the use of OSM. It is no coincidence that most of our trainings this summer focused on teaching OSM to members of NGOs, local government, universities, and civil society organizations. We even took a trip to Leyte, one of the provinces most affected by Typhoon Haiyan, and held an OSM workshop there. Our hope is to foster disaster resilience in the Philippines by building a large community of OSM mappers. Ideally these mappers will fill in the gaps by mapping more roads, evacuation centers, health facilities, and disaster aid.
 
Recommendations
In order to get a more comprehensive picture of the humanitarian response to Typhoon Haiyan, it is imperative that all donors report their contributions to open data platforms such as FAiTH or IATI. It is also important that donors include more detailed project descriptions and disclose the amounts of not only the funds committed but also the funds disbursed.
 
To determine whether the funding received in response to Typhoon Haiyan had any impact on the quality and quantity of disaster infrastructure in the top ten flood risk and storm surge provinces, we need more detailed information about said infrastructure. For example, it would be useful to know when the health facilities and evacuation centers were built.
 
Lastly, we recommend holding more OSM, GIS, and geocoding workshops in the Philippines. Doing so will build capacity among NGOs, universities, government agencies, and civil society organizations to use geospatial tools, foster the growth of the local OSM community, and make disaster resilience in the Philippines a more realistic and sustainable goal.
​

Daniela Hernandez Salazar is a Graduate Research Fellow and Task Team Leader for IPD’s Governance and Corruption team. She is a first-year master's student seeking a dual degree in Global Policy and Latin American Studies at the University of Texas at Austin. Daniela was an AidData Summer Fellow in the Philippines during the summer of 2015.  ​

​​Innovations for Peace and Development (IPD) at the University of Texas at Austin was established in January 2013 to produce and disseminate rigorous, policy relevant research to promote innovations in global peace and development. With research teams ranging from Climate Change to Conflict, the lab casts a wide net for their analysis of pressing development issues in today’s world. An issue of particular interest to the lab is evaluating the use, uptake, and impact of open aid data. While the methods of evaluation in this subject are nascent, studies are emerging on the impact of open data use throughout developing countries. Recently, the IPD’s Open Aid Team conducted a review of the research provided by Open Data for Developing Countries: Exploring the Emerging Impacts of Developing Countries. Our research examines the types of studies undertaken and limitations and findings associated. Based on these findings, we highlight the gaps in the current research and offer recommendations for the necessary innovative leaps in evaluation and research.

Current Evaluations
Exploring the Emerging Impacts of Open Data in the Developing Countries is a multi-country, multi-year study led by the World Wide Web Foundation to understand how data is being put to use in different countries and contexts across the developing world. According to the Open Data Network website, “these case studies examine initiatives, the governance challenges they propose to address, and emerging outcomes and impacts from the application of open data in these contexts. The project is also developing cross-cutting data collection instruments and analysis approaches to help explain if and how open data is bringing change to developing countries.” Together, these studies will help improve developmental outcomes of open data initiatives.

Types of Studies
The Open Aid Team at IPD analyzed 24 of the 26 studies included in the Emerging Impacts Project. The two unanalyzed projects were a Open Data Research Symposium and upcoming synthesis paper. Of the 24 studies, 7 utilized a mixed methodology approach, 13 utilized a qualitative approach, and 1 utilized a quantitative approach. The studies spanned research projects in 16 different countries.

Findings
There were several themes present across the analyzed studies. The findings are grouped into three categories: limited accessibility of the data concerns over social exclusion, and the lack of strong causal linkages between supply and impact.

Access to open data information remains an issue across the majority of research environment. Two roadblocks hinder achieving the ideal level of access to data. First, there is a lack of intermediaries between the suppliers of the data and those intended to be demanders. The second issue is that the current technology used to access open data is neither user friendly nor sensitive to low Internet connectivity. Several studies highlight the need to more effectively streamline data in order to achieve greater accessibility.

A handful of studies question the effectiveness of initiatives that are solely digital and are published and disseminated only in written form. These studies critique the reliance on digital capacities and digital mediums in order to use and access open data. By only tailoring the openness of data to an elite and literate portion of the population, will marginalized groups be further ostracized by their lack of access to pertinent information? Essentially, there is a need to move forward with caution. We risk further deepening the digital divide if we do not work to bridge this gap. 

Several studies cited a difficulty to develop strong evidence of impact due to the early stage of development of the data itself within the respective contexts. For example, the studies within both Nepal and India alluded to the trouble of correlating supply and impact due to nascent open data systems. The studies focused on process and implementation measures as opposed to linking impact with supply.

Recommendations for Future Research
Based on our analysis of the Emerging Impact Project, we offer the following recommendations for future research on the use and uptake of open data.

Press for Innovative Evaluation Methods
One of the limitations to the current research undertaken in this subject is its inherent qualitative nature. Though the studies included in Emerging Impacts Project span from Brazil to Bangladesh, they are woven together by a qualitative approach that utilizes case studies and interviews. Typically, non-random, self-selected sample groups are chosen for such cases. Additionally, the majority of cases have an isolated focus of the use and uptake of data at the national level.

It is well understood that evaluation methodology for this type of subject matter is intricate, complex, and unknown. Scholars and stakeholders should continue to discuss how to evaluate open data initiatives while thinking strategically about what constitutes impact within a specific environment. Two questions must be asked. What type of impact is open data looking to bring about within a society? How can we begin to quantify or measure those impacts within an evaluation? This subject arena should continue to be explored in great detail. Though it has yet to be systematically executed, randomized control trials have potential to show that the provision of a specific type of open data to relevant stakeholders may change perceptions on issue prioritization or supplementary decisions. Yet in order to execute such a novel implementation of an RCT design, several looming issues must be addressed. Integral in the discussion are topics such as establishing a reputable control group, arriving at measurable indicators, and understanding how to measure spillover within a close-knit environment such as governmental agencies.

Focus More on the Data Supply Chain
Future research should place more focus on the role of intermediaries within the open data ecosystem. At this point in the data revolution, research on the supply side is well established. We now need to work on establishing intermediaries to link the supply of data to the use of data.  The next stage of research should address methods for creating these intermediaries, determine which intermediaries are the most influential and successful, and explain why. It is crucial that the research better understands how to overcome the challenges to linking actors within the open data supply.
In addition to the need for intermediaries is a need to understand where the injection of open data systems fits within existing structures. When researching the use and uptake of open data, studies should concentrate on understanding the current decision making processes and under which conditions open data information is most relevant. Potential questions to ask include:

• Where within the decision making process is it most relevant for actors to consult newly available open data?
• How can the new systems build off of existing platforms to ensure greater use?
• Where within the decision making process should open data be placed to achieve optimal efficiency?

Though this research will be context and country-specific, it can serves as a valuable stepping-stone for unleashing the potential impact of open data systems.

Focus More on Building Capacity
A large portion of the open data use literature discusses the difficulties developing countries face in establishing, maintaining, and financing strong data ecosystems. A major barrier to unlocking the potential of open data will be changing current behavior and habits of the intended users of data. Research should focus on what is necessary to motivate data use as well as what motivates current data use. These lessons should then be applied across the board. Researchers must investigate the conditions and circumstances under which the uptake and use of data is most likely and explain why. In addition to modifying behavior, if countries do not have the institutional and infrastructural capacities, uptake of data will continue to trudge along. Countries must have both a strong technological bandwidth as well as a strong human bandwidth. Because the impact of open data is contingent upon these two strengths, attention should be given to building such capacities.

Discuss Strategic Action Plans and Feasibility
Current studies lack the necessary action plans needed to build and foster use of open data systems. Little focus has been placed on calculating cost benefit analyses for the recommendations of greater uptake. While it is important to understand what is needed to foster greater uptake and use of open data, it is equally important to understand the feasibility of such implementation measures. Studies undertaken in countries must not only note what should be done but what can be done. Research must take into account the political willingness and the resource capacities of the specific context at hand.  It is crucial that an induction of feasibility and strategy accompany recommendations. Additionally, as major development partners such as the UN and the World Bank call for a data revolution, their development strategies and country development plans should embody a strong commitment to bolstering and funding these necessities.

Promotion of Mobile Technology for Data Collection
Another factor of importance to consider is the possibility of further mobile technology options for data collection and dissemination. Given the growing uptake of mobile devices in developing countries and their already high usage in many places, this remains a not fully tapped strength in the efforts for data transparency. As open data continues to be a growing field, gathering and distribution of relevant data through mobile phones via SMS messages, smartphone apps, call-in voice programs and other mechanisms could be the tool that will close the gap. Real-time data collection and dissemination of needed data on demand would significantly change the impact of any open data efforts.