- A team of researchers trained an artificial intelligence tool using a million photos of disasters to recognise disasters and incidents that need human assistance.
- From using it at an ad hoc level in the past, deep learning tools can now systematically analyse incidents to gauge a pattern for future use as well.
- In using social media images for disaster management, challenges include training AI tools, navigating misinformation and certain social media policies.
When disasters strike, social media gets flooded with images, warnings and calls for help. Many of the posts are sources for relevant information from disaster sites and the data can help understand the progression and aftermath of a disaster. But manually segregating and analysing the data is a time consuming, costly and often inefficient process.
While deriving useful information, a new study by a multi-country research team presents a large-scale dataset and explores how to automate information processing about natural disasters from social media images. The study used one million photos of natural disasters and other types of incidents that need human intervention, to train an AI (artificial intelligence) system to recognise similar disasters and incidents automatically.
Training AI to spot signs of disasters in images
One of the authors of the study Agata Lapedriza, head of the Artificial Intelligence for Human Well-being Lab at eHealth Center, Barcelona, said that the motivation for their research was the urgency with which humanitarian organisations need to arrange for relief efforts during a calamity. There is existing evidence that social media networks can be an interesting source of information during disasters since affected people tend to post photos and information on what is happening. “The idea, therefore, was to design tools to automatically filter images from social media and detect natural disasters and other types of incidents,” she said.
The multi-label dataset studied had 9,77,088 images, with 43 incident and 49 place categories. Explaining the research process, Lapedriza said that the images collected from the internet were labelled appropriately as disaster-related or not and then an AI system (deep learning model) was trained to detect disasters from images obtained from social media platforms, such as Flickr or Twitter (currently known as X). “Within these image sets, our deep learning model could detect disaster images. We found that there was a correspondence between the incidents detected in the images and specific incidents found in the records of Nepal and Chile earthquakes in 2015, or Bangladesh floods in 2017,” she said.
Lapedriza hopes that the tool that they have developed would open doors for humanitarian organisations to be better informed about ongoing disasters, which could improve the management of humanitarian aid when it is needed.
From using such tools at an ad hoc level like sending warnings to residents and travellers of a place at the first sign of a potential disaster, deep learning tools can now systematically analyse the incident to gauge a pattern for future use as well, said Muralee Thummarukudy, an expert in disaster management and director of G20 global initiative coordination office at the United Nations Environment Programme. He sees a need and potential in scaling up social-media-driven disaster risk assessment, management and predictions in India since the country has the advantage of a large population and a deep penetration of social media even in rural areas. “The huge population and data set and cheap data are advantages but the potential remains untapped. Disaster management is still considered a philanthropic activity. The governments should take an interest in this and tap the potential in India which can make the country a role model (in using large data sets for disaster prediction and management) for the rest of the world,” said Thummarukudy.
Crowdsourcing data for flood management
A separate study by IIT Mumbai, which is yet to be published, explored the feasibility of using crowdsourced information as a data source for flood forecasting and warning. The results confirmed the dependability of social media posts to accurately reflect the ground situation.
At first, the researchers retrieved flood-related data from Twitter (currently known as X) using location information to generate flood maps for heavy rainfall events in the past. The validity of the retrieved data was confirmed by comparing it with information via volunteers (volunteered geographic information or VGI) which is more accurate but less abundant. The Twitter data is cross-verified with the Height above the Nearest Drainage (HAND) map — a scientific measurement for producing flood inundation maps — which serves as a proxy for elevation. The study found that while extreme rainfall events were increasing in frequency, recent Twitter-based information showed a decrease in flood reporting. This was due to effective mitigation measures implemented at various flood hotspots, a fact verified by local volunteer information.
The study demonstrated the usefulness of data aggregated through social media in real-time flood monitoring and forecasting. “Social media has a lot of data but not many are useful. Sometimes the information may be incomplete or it is not geo-tagged properly that reduces the efficacy of it. Constantly changing Twitter (currently X) policies was another big hurdle in using the data,” Subimal Ghosh, a professor at the department of civil engineering at IIT Mumbai who was a part of the study, said.
Challenges include tackling misinformation
For Lapedriza and her team, the main challenge was the inability of the trained AI model to positively detect the incident or the disaster. For example, for car accidents, the model was detecting images with cars but not necessarily those in accidents. To plug that, they incorporated the idea of class positives and class negatives and trained the AI model to detect the class positives only. “For example, for the class wildfire, we will have a collection of images of wildfires (class positive) and a collection of images that do not show wildfires (class negatives). The interest of the class negatives is to show to the AI model images that might look like wildfire, like say, a fireplace, but that are not showing wildfire,” she explained.
In a 2021 review of social media data use in disaster management, the reviewers point to Twitter as a critical information provider with the temporal and spatial information extracted from the medium as helpful in supporting decision-making in disaster management. The review notes that geo-location identification and analysis are key research challenges of the spatial perspective in disaster management. Social media content, along with temporal information, including posting time and event time, can be used to facilitate disaster management in several ways.
One concern experts have is that since the social media landscape is riddled with misinformation or fake news, it could come in the way of the authenticity of the data. Experts such as Thummarakudy say they feel that governments should be actively involved in setting up a regulatory framework to control fake news during disasters. “Eventually, the AI algorithms will be able to catch up and become powerful enough to weed out misinformation. But better regulation always helps to make people aware of the consequences of spreading misinformation,” he said.
Read more: Study finds limited outreach of disaster-related mobile apps
Banner image: Photo of flood-affected New Delhi in 2023. New studies show social media can play a crucial role in disaster prediction, management and recovery. Photo by Shaz Sayed/Mongabay
