PauletX

https://youtu.be/4LbYm7V8CQg

https://nasa-space-apps-challenge-2021.vercel.app/

𝐏𝐫𝐨𝐣𝐞𝐜𝐭 𝐃𝐞𝐦𝐨:

Share your code: The Github code of the Track the plastic, where you can find the Computer Vision algorithm, webpage code, virtual assistant code and information.

Solution as web platform: It is a beta version, due time constraint it was not possible to develop it fully. But it is feasible to achieve if provided additional time: Track the plastic webpage

The solution allows the user to use three main functionalities, which can be accessed intuitively or through a virtual assistant: 

Track the plastic info: Information of the project Track the plastic (files) https://drive.google.com/drive/folders/18954DTi_tbet0-AQYv57I49NFUUdPmSh

𝗦𝘁𝗮𝗿𝘁 𝘁𝗿𝗮𝗰𝗸𝗶𝗻𝗴 :

The user can upload images of marine plastic debris (for now in coastal areas) that they found near their location through the virtual assistant. You can upload a photo in which you identify debris presence. Then the image is analysed by a Computer Vision algorithm based in deep learning and object detection (YOLO framework) in order to detect 5 types of debris, such as: Glass, Plastic, Metal and Fishing gear and other trash(e.g. wood, tyres), and calculate the quantity of trash found. All this information gathered from the previous phase is stored in a as-a-service database (DB2) that was modelled to keep relevant information. Finally, this information will be consumed by the dashboards and the heatmap.

Image: Object Detection of the plastic debris

𝐅𝐢𝐧𝐝 𝐢𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧:

The virtual assistant consumes information from the database, which is fed by NASA resources. Moreover, it has a scraping algorithm that enables responses of more detailed information related to pollution, debris to provide a holistic view of the pollution. To summarize, the virtual assistant will make information accessible to citizen scientists, remote-sensing experts, as well as policy makers and regulators in order to use this information to effectuate change or be part of the change (Plastic debris community).

Image: Virtual Assistant based in NLP that enables the object detection and storage of image database

𝗠𝗮𝗽 𝘁𝗿𝗮𝗰𝗸𝗶𝗻𝗴:

The virtual assistant asks you to redirect to the heat map that is inside the web page so that the user can identify the main points of contamination and obtain information from these. The information displayed by the map has also been fed by the previous identification of plastics in remote satellite images provided by NASA resources within the HawkEye project. The user can identify relevant information and share it on Twitter. Furthermore, all data collected from users via the Start Tracking functionality will be consumed and displayed in the map, showing markers on the affected zones as a call for action to local communities. 

Image: Map that enables plastic debris tracking

𝗛𝗼𝘄 𝗱𝗼𝗲𝘀 𝗶𝘁 𝘄𝗼𝗿𝗸?

The solution has 4 components which are:

• The implementation of a web page that shows in real time the activity of marine plastic pollution in the world can through dashboards and a heat map. In addition, once relevant information has been identified, users can share it with their contacts through Twitter.
• A virtual assistant that allows users to have a simple and effective interaction to find current and real information on marine pollution. In addition, it allows users to have a means to upload the contamination images that they identify.
• A computer vision algorithm, which allows to identify and classify the presence of plastic waste within marine surfaces.
• An as a service database, designed to store the information that users collect on marine pollution and the results of computer vision algorithms.

𝗕𝗲𝗻𝗲𝗳𝗶𝘁𝘀:

• Create a visualization database based on AI/ML algorithms that will aid in classifying and detecting these plastics using remote sensing data from NASA and users (debris community)
• Encourage people to clean debris from coastal areas through Social Network (Twitter) in order to reduce the amount of contamination in the coastal areas.
• Empower people in making little changes that will impact the levels of pollution of their community through the tracking and classification features of the virtual assistant.

Image: Desktop Interaction

Image: Responsive Interaction

Image: People cleaning coastline's debris (Debris Community)

𝗪𝗵𝗮𝘁 𝗱𝗼 𝘆𝗼𝘂 𝗵𝗼𝗽𝗲 𝘁𝗼 𝗮𝗰𝗵𝗶𝗲𝘃𝗲?

Right now in the ocean there are 5.25 trillion macro and micro pieces of plastic. We want to reduce in one year the 0.5% of plastic in the coastal areas. To achieve that we want to generate awareness to the citizens through “Track the plastic”. In the webpage people will have access to actual information of marine plastic debris, they also can search the nearest plastic debris from their location and they can share that information through social media. We plan to create an ever growing community and a movement in which the people with the same attitude can take action and get reunited to do some activities like cleaning their coastal areas.  (Ferris, 2021)

Left Image: Debris in the coastline in before debris community clean activitiess

Right Image: Debris in the coastline in after debris community clean activitiess

𝐒𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐅𝐥𝐨𝐰:

Image: Project functionality flow chart of the project

𝐒𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞:

Image: Project functionality flow chart of the project

1. SeaHawk/HawkEye (https://oceancolor.gsfc.nasa.gov/data/hawkeye/):

“The Hawkeye instrument, flown onboard the SeaHawk CubeSat, was optimized to provide high quality, high resolution imagery (120 meter) of the open ocean, coastal zones, lakes, estuaries and land features.”

We use these images to identify marine pollution through the computer vision algorithm created since they are images that are 120 meters away.

2.OB.DAAC API (https://oceancolor.gsfc.nasa.gov/data/download_methods/)

“ API to select parameters for your data query and return results to the screen. File search can also be preformed via command line ”

We use this API in order to gather images from a satellital image database in order to use this for visualization purposes of the lakes, ocean in our dashboard.

3. OFFICE FOR COASTAL MANAGEMENT (https://coast.noaa.gov/states/fast-facts/marine-debris.html)

“Marine debris is a global problem. The mission of the NOAA Marine Debris Program is to investigate and prevent adverse impacts from marine debris.”

This type of general and statistical information helps the virtual assistant to answer questions related to contamination.

4. ECCO Sea Surface Height (https://search.earthdata.nasa.gov/search/granules?p=C1990404813-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&fdc=Physical%20Oceanography%20Distributed%20Active%20Archive%20Center%20(PO.DAAC)&ac=true&tl=1633287260.168!3!!) 

Data of daily-average dynamic sea surface height, that presents estimations of circulation and climate of the oceans in a dynamically way. This data helps to understand the rising sea levels and pin point the main problematic zones.

This numerical data will feed the real-time dashboard that shows all info related to our oceans. Additionally, the virtual assistant will use this to answer related questions.

5. ECCO Ocean Temperature and Salinity (https://search.earthdata.nasa.gov/search/granules?p=C1990404795-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&fdc=Physical%20Oceanography%20Distributed%20Active%20Archive%20Center%20(PO.DAAC)&ac=true&tl=1633287260.168!3!!)

This dataset contains monthly-average ocean potential temperature and salinity. This data gives us an understanding of the always evolving ocean, sea-ice, and surface atmospheric states.

This numerical data will feed the real-time dashboard that shows all info related to our oceans. Additionally, the virtual assistant will use this to answer related questions.

6. ECCO Ocean Velocity (https://search.earthdata.nasa.gov/search/granules?p=C1990404823-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&tl=1633287623.802!3!!&fst0=Oceans&fsm0=Ocean%20Circulation)

Contains daily-average ocean velocity data sent by satellite altimeters. This data shows the velocity of the oceans in their different sections.

This numerical data will feed the real-time dashboard that shows all info related to our oceans. Additionally, the virtual assistant will use this to answer related questions.

It has been a challenge to find a solution in 48 hours, however we have a very enriching feeling because we know that we are creating a solution that generates global impact. We are very inspired by being able to take care of our planet. It helps us learn things about maritime pollution, climate change because it is important to know about the house where we live in order to take care of it and protect it.

We think it has been a good opportunity to learn about a problem of great impact and think about solving something that helps all of us (humanity). We also feel that we are inspired by the desire to do something for our planet and to be able to generate some change through the unity of people or as a group. Our approach seeks to attack the origin of where the waste originates (being also where most of these are) the coastal areas. In addition, during our research we found that NOAA considers river cleanup to be a difficult and costly problem and that is why we seek to attack coastal areas through the united effort of the community. We would like to thank NASA for giving us the opportunity to participate in this challenge of great impact.

To find the solution to this challenge, we really believe that we all can be an agent of change and use technology to quickly identify and monitor pollution . We thank NASA and NASA Space Apps Challenge Lima for the opportunity to participate in this great challenge.

From the beginning there was a good synergy in the team, which allowed us to quickly understand each other's ideas and to be able to find what role each member would play. In this way, we were able to divide the work and support each other in order to move forward efficiently and achieve the final result.

We had good communication between the members, which allowed us to have a more effective development, and that we could address problems and solve them quickly.

Image: Team members

𝐃𝐚𝐭𝐚:

1. C. Feldman. (2021). SeaHawk/HawkEye. Retrieved from: https://oceancolor.gsfc.nasa.gov/data/hawkeye/
2. C. Feldman. (2021). Search and Download Methods. Retrieved from: https://oceancolor.gsfc.nasa.gov/data/download_methods/
3. OFFICE FOR COASTAL MANAGEMENT. (2021). Marine Debris: https://coast.noaa.gov/states/fast-facts/marine-debris.html
4. Condor Ferries. (2021).Information of the plastic pollution in the oceanhttps://www.condorferries.co.uk/plastic-in-the-ocean-statistics
5. ECCO Consortium, Fukumori, I., Wang, O., Fenty, I., Forget, G., Heimbach, P., & Ponte, R. M.. (2021). ECCO Ocean Temperature and Salinity - Monthly Mean 0.5 Degree (Version 4 Release 4). EARTHDATA NASA.https://search.earthdata.nasa.gov/search/granules?p=C1990404795-POCLOUD&pg[0][v]=f&pg[0][gsk]=-end_date&fdc=Physical%20Oceanography%20Distributed%20Active%20Archive%20Center%20(PO.DAAC)&ac=true&tl=1633287260!3!!
6. ECCO Consortium, Fukumori, I., Wang, O., Fenty, I., Forget, G., Heimbach, P., & Ponte, R. M.. (2021). ECCO Ocean Velocity - Monthly Mean 0.5 Degree (Version 4 Release 4). EARTHDATA NASA. https://search.earthdata.nasa.gov/search/granules/collection-details?p=C1990404823-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&tl=1633287623!3!!&fst0=Oceans&fsm0=Ocean%20Circulation
7. ECCO Consortium, Fukumori, I., Wang, O., Fenty, I., Forget, G., Heimbach, P., & Ponte, R. M.. (2021). ECCO Sea Surface Height - Daily Mean 0.5 Degree (Version 4 Release 4). EARTHDATA NASA. https://search.earthdata.nasa.gov/search/granules?p=C1990404813-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&fdc=Physical%20Oceanography%20Distributed%20Active%20Archive%20Center%20(PO.DAAC)&ac=true&tl=1633287260.168!3!!

𝐓𝐨𝐨𝐥𝐬:

• LabelMg: for the labeling and annotation of the images used to train the YOLO detection model. 
• REST API: Twitter Developer API
• React
• Prototype design: Figma and AdobeXD

𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞𝐬:

• Python
• Javascript

𝐒𝐨𝐟𝐭𝐰𝐚𝐫𝐞:

• IBM DB2
• IBM Watson Assistant

#ai #plasticdebris #ml #webpage #mobile #tracking #map #computervision #objectdetection #nlp

This project has been submitted for consideration during the Judging process.