A Crowdsourcing Pairwise Comparison Method for Recognizing Duplicated Disaster Reports

Disaster reports play an important role in acquisition of information for disaster responses. In the disaster emergency operation, decision-makers need to decide the allocation of labor and resources according to disaster reports. However, some reports are duplicated so that decision-makers might allocate resources again on the same event. This causes not only higher risk of losing relief members'' lives but also short of relief resources. To resolve the problems, this research addresses two issues: (1) develop a crowdsourcing platform for recognizing duplicated disaster reports; (2) develop an evaluation method for crowdsourcing requirements. We developed a crowdsourcing pairwise comparison method to systematically crowdsource the work of recognizing duplicated reports with lower workload. Specifically, results of comparison tasks are determined based on validated feedback from crowds. Then, we raised a crowdsourcing requirement evaluation method, which quantizes the workload of crowdsourcing tasks and the crowd individual capacity for the estimation of required number of crowds in our project. Finally, we established an online platform P+ and applied it during a real disaster, Typhoon Dujuan in 2015 in Taiwan. Statistically, 5,656 disaster reports were collected in 30 hours, and 1,980 tasks were assigned to 93 registered people gathered from Facebook. The crowds could complete a comparison task in 25 seconds. We found that reports grouping was effective in decreasing over 90% crowdsourcing workload in P+. Besides, the estimation of crowd individual capacity had the accuracy of -1.45% error. These results show that our platform has potential for the recognition of duplicated reports. Moreover, we had an important observation that disaster reports had an exponential growth but volunteers increased linearly. Therefore, it is necessary to divide reports into more groups with lower workload. Another solution is to determine the factors of increasing crowds in social media. Further, we suggest to regularly gather active volunteers in social media for more crowdsourcing capacity. In conclusion, P+ is a feasible crowdsourcing platform, and the crowdsourcing requirement evaluation method helps set up terms of project goals.