AI MEETS CLIMATE ACTION
Smart Voice Reporting System: AI for Climate-Induced Loss and Damage Reporting

1. Introduction

As part of Oxfam’s commitment to address climate-induced challenges, a Loss and Damage (L&D) Dashboard has been developed which is a web-based platform designed to explore, document, and analyze evidence of climate-induced loss and damage affecting vulnerable communities.  At the heart of this platform lies a concise social survey form, offering a critical space for individuals to report the specific losses and damages they have experienced due to climate change. These self-reported accounts are synthesized and analyzed, providing a holistic understanding of the climate crisis by integrating personal narratives with geographical and earth observatory data. Recognizing the need to make this vital tool even more inclusive and accessible, Oxfam aims to upgrade the system by introducing self-reporting mechanism by the Loss & Damage affected communities. However, the key challenge is that the L&D affected communities do not have enough literacy and access to digital device and internet to self-report their L&D. However, in terms of inclusion, evidence and broader negotiations it is crucial that self-reporting mechanism functions well and for all irrespective to the people who are climate vulnerable and left behind. Therefore, Oxfam has come up with an innovative idea to adopt Artificial Intelligence (AI) with its existing Paroli system to facilitate the self-reporting system through voice recognition. The AI based voice recognition which would eventually complementing the self-reporting system would require some protocols and developments within the existing L&D dashboard to be capable of declaring it a self-reporting system.  Oxfam therefore, to facilitate this developments, is launching an exciting competition to inspire students to develop innovative solutions around the AI based voice recognition complementing the self-reporting for the dashboard. 

The competition challenges participants to design a reporting mechanism that enables individuals to share their L&D cases through toll-free phone calls, making the platform accessible even to those in remote areas or with limited digital literacy. An AI-powered system for collecting, analyzing, and reporting data is being inspired., The proposed solutions will incorporate a machine-learning model capable of translating local dialects into structured inputs that automatically populate the already built social survey form and store the captured data. Phone conversations will be recorded in Bengali, and the system will extract relevant information from these conversations, including answers to predefined questions. The system will employ speech-to-text (ASR), question detection, information extraction, and natural language processing (NLP) techniques to process the conversations and generate the reports.

To be more specific, the proposed solution will:

• Automatically record and transcribe phone conversations in Bengali to fill up survey form.
• detect and extract key information related to climate change losses and damages from transcribed text.
• classify questions and responses as required by the system.
• generate structured reports based on the extracted information.
• store the extracted information into an excel/csv file for usability.
• create a scalable, automated system that can handle multiple dialects of Bengali.
• system should have provision of incorporating new data or section.

2. About Loss and Damage Dashboard and its Importance

The topic of loss and damage (L&D) has increasingly become a focal point, highlighting the connection between climate change (CC) science and policy. The United Nations Framework Convention on Climate Change (UNFCCC) acknowledges loss and damage as a key element in climate change negotiations, emphasizing the need for financing mechanisms to assist impacted communities.

Climate-vulnerable Least Developed Countries (LDCs) face significant challenges in their efforts to secure and expand the Loss and Damage (L&D) fund. Wealthier nations, often responsible for financing this fund, frequently resist these efforts, engaging in denials and debates. To avoid accountability, they tend to conflate climate-induced loss and damage with conventional losses, undermining the unique and urgent need for compensation. LDCs often present aggregated data and case studies to advocate for the fund, striving to illustrate the scale of the issue and draw global attention. However, a stark contrast exists between the impact of such reports and the narratives shared directly by L&D victims. Victims' firsthand accounts not only evoke stronger emotional responses but also provide concrete evidence of the real and devastating impacts of climate change. Bridging this reporting gap could significantly enhance the urgency, credibility, and effectiveness of efforts to secure equitable support.

The L&D Dashboard is a progressive tool, which shows spatiotemporal information to make aware of the L&D hits over time. For example, if an individual reports an L&D case of agricultural land loss due to climate change, the L&D dashboard would register the case with exact geographic information of the land that has transformed from agriculture to other land cover (e.g., river). The dashboard would then a) analyze data of that geographic location and visualize the trend of climate change, b) analyze the historical satellite remote sensing data to prove that the agricultural land existed before L&D happened using state-of-the-art satellite remote sensing and Geographic Information Science (GIS). So, the L&D dashboard can be an inclusive proof of reality which many developed countries pursuable treat as fiction. This dashboard can support negotiators, researchers, activists, media, etc. from many levels to recognize and address the disproportionate impact on vulnerable communities. It will support equity in climate policies so that the least responsible for the climate crisis yet most impacted are not unfairly burdened.

3. Why the Competition

The aim is to ensure that climate-affected individuals, regardless of their technological or literacy constraints, can easily and directly report their experiences. By integrating the proposed solutions into the Loss and Damage Dashboard, the platform will be better positioned to amplify the voices of vulnerable communities, drive meaningful action on climate change, and support a more inclusive approach to documenting and addressing the far-reaching impacts of the climate crisis. To actualize this aim, Oxfam believes the solutions can be best sought by inspiring innovation through a competition among the young minds who are visionary and believes in integration of digital technology to keep up with the pace of the world, at the same time, will work to bridge the digital divide by making solutions more accessible and cost effective. Participating students will develop a system that integrates with existing databases or dashboards for real-time self-reporting and analysis as stated above. Winners will have further opportunity to work on the project with Oxfam as Fellows through a separate provision.

4. Plan

• Launch of the Competition
• Online FAQ Session
• Application with Concept Submission Deadline – 31 March 2025
• Initial Selection of 10 Shortlisted Applicants/Team
• Winner Selection and Award Ceremony
• Onboarding of selective potential winners as consultants for Phase 2

5. Eligibility

All university students will be eligible for the competition. Participants can apply individually or in groups.

6. Scope of the Competition

• Automatic Speech Recognition (ASR) for Converting Phone Conversations into Text for predefined section of form

Implement an Automatic Speech Recognition (ASR) system to transcribe voice recordings from climate-affected individuals into accurate text for the specific section of the form. The system should be optimized for quality and accuracy, ensuring transcription of varied accents and speech clarity. This step is essential for making voice-based data accessible and actionable.
 
Expected Outcome: Fully operational ASR system capable of transcribing phone conversations into text.

• Identification of Climate-Related Questions Using Natural Language Processing (NLP) 

Use Natural Language Processing (NLP) techniques to detect and classify specific climate-related questions within the transcribed text. These questions may include inquiries about types of climate-induced damage, location, affected areas, damage type and quantity.Account for different Bengali dialects and ensure the system works across these variations.

Expected Outcome: Custom NLP model for identifying and classifying climate-related questions in transcribed text.

• Structured Report Generation from Extracted Information 

Develop a system to generate structured, readable reports from the extracted information. These reports will summarize the climate-related losses and will be presented in an accessible format for stakeholders such as Oxfam staff, policymakers, and local communities. Use Named Entity Recognition (NER) and custom models to extract locations, dates, damage types, and quantities mentioned in the responses.

Expected Outcome: Report generation system that outputs structured reports in formats such as Excel, JSON, CSV, or PDF.

7. Criteria for Evaluation

8. Benefits

• Winner/Winning team will be awarded BDT 1,50,000 (One lac fifty thousand taka)
• Runner up team will be awarded BDT 1,00,000 (One lac taka)
• Third position will be awarded BDT 85,000 (Eighty-five thousand taka)
• Award of Potential – BDT 30,000 (Thirty thousand taka) - Up to 3 teams

9. Proposal Submission 

The proposal should be submitted to: hrbd@oxfam.org.uk by 31 March 2025, 11:59 PM. All participants must submit their proposals using the email subject line "Smart Voice Reporting System - Applicant's Name". The guidelines of the proposal are as follows:

• Cover Page Requirements

Mode of Participation: Specify whether the submission is from an individual or a group.
Participant Details: Include name(s), email address(es), and phone number(s) for all participants.
Affiliation: Mention the institution/organization, if applicable.

• Model Sketch and Description

Sketch of the proposed model: Digital illustration or software-generated diagram and description of the proposed model.

10. Phase 2: Future Scopes after the Competition

The competition will identify the best solution. Oxfam will seek financial proposals from the winning teams and other potential applicants as per future vision of the project and collaborate to develop scalable solutions.

Methodology: To scale up the work and integrate it into the existing dashboard, selected winners will adopt the following methodology for solutions

• Data Collection:
• Gather phone recordings of conversations with individuals reporting climate-related damages.
• Ensure diverse data is collected from various Bengali-speaking regions, including dialects such as Sylheti, Chittagonian, and others.
• Preprocessing:
• Convert recorded audio into text using ASR models.
• Clean and preprocess the transcriptions to remove noise, irrelevant information, and improve accuracy.
• Model Development:
• Train question detection models using supervised learning methods to identify climate-related questions from transcriptions.
• Fine-tune NER models for Bengali to detect key entities such as location, quantity, damage type, and event date.
• Implement dialect handling by training models on data from different Bengali dialects.
• Report Generation:
• Implement a structured format for reporting and storing extracted data.
• Automate the generation of reports and ensure they are compatible with different stakeholders' needs (e.g., authorities, NGOs, local communities).
• Testing and Validation:
• Test the system using a separate set of recorded phone calls to evaluate the accuracy of question detection, information extraction, and report generation.
• Fine-tune models to improve performance, especially for dialects and complex language patterns.
• Deployment: Deploy the system on a scalable infrastructure that can handle continuous phone conversations and generate reports in real-time.

Specific Deliverables:

• Transcription Model: An ASR model capable of transcribing Bengali phone conversations.
• Question Detection Model: A machine learning model trained to detect climate-related questions in Bengali.
• NER Model: A model for extracting key entities like locations, damage types, and dates.
• Structured Report Templates: Predefined report formats for documenting the extracted information.
• Deployment Platform: A web-based or cloud platform for managing phone call recordings, processing data, and generating reports.
• Testing and Performance Reports: Metrics on system performance, including accuracy of question detection, entity extraction, and reporting.