Equipe Minerva

Salvador, Bahia State | Leveraging AI/ML for plastic marine debris

Application of CNN (Convolutional neural network) and Computer Vision to identify marine debris.

High-Level Project Summary

Scientists estimate that by 2050 there should be more plastic in the ocean than fish, and with that there is a “race” among scientists to avoid situations like these. The idea consists of an artificial intelligence that, together with the computer vision process, can help in this process of cleaning up marine debris. Through a long machine learning training, the AI would be to know how to identify by the image of a camera that would be attached to a device the MarinhI.A and when the frames were captured, these images would be sent to a central that notifies the team. divers what are the exact points to carry out the collection, optimizing the time and making searches more efficient.

Link to Project "Demo"

https://drive.google.com/file/d/17b4pNiBzeQs1v9BGECdc0JhYiIRJ-0wo/view?usp=sharing

Link to Final Project

https://drive.google.com/drive/folders/1dUSu5af1alwbg43tHXrHp237gTNNvmxB?usp=sharing

Detailed Project Description

To create a deep neural network takes time and a varied database, our team did not find any database, especially dataframes, so that we could train the neural network, so we outlined the step by step of how it would be created .

1.1 - DATA COLLECTION

this step will consist of collecting data with cameras that can record images underwater, going through a long path and recording images in different ways (RGB, which are color and NIR (Neir infrared, which are infrared frames), preferably in places that we know there is garbage, and this camera will record videos and divide these videos into frames, it will need a large set of frames and they will be organized in a dataset;

1.2 - DATA ANNOTATION:

This step will be done through people who will look at the frame images and through the CVAT data annotation platform for computer vision, they will mark garbage that the photos contain, then labeling the data;

1.3 PRE-PROCESSING OF DATA:

This step depends on an analysis of how the frames are to reduce whether some kind of pre-processing will be necessary to facilitate the visualization of the garbage, processing the data and analyzing the annotations;

1.4 PROCESS DATA:

If you need treatment, treat the data and create a set with NIR, that is, infrared image dataset;

2.1 Construction of the network and its structure

This step involves the creation of a convolutional neural network structure based on existing networks such as ResNet or Inception to build a network capable of identifying garbage in the photos;

2.2 NETWORK TRAINING AND PARAMETER TESTING:

This step consists of training the network and generating models with different parameters to define the best combination of parameters, making GRIDSEARCH, a technique that combines parameters and returns the best combination within the proposed options, so it will be possible to improve the results of the network, we intend to do this step with 80% of the dataset, as we will need a large amount of data;

2.3 VALIDATION:

In addition to training, the Validation will be carried out in order to observe results, with a view to considering network overfittings or underfittings to know how we can minimize these problems and improve the quality of the project

2.4 TEST WITH TREATED IMAGES

Creation of datasets with kernels, thresholds and/or effects on the images to redo training and validations and observe if there was an improvement or deterioration in the result;

2.5 Select the best model and the best training process based on the observation of the parameters, define the best model and which will be used for production, in addition to observing the results of the image treatments and which is best for the convolutional neural network to work;

3.1 NEW DATA COLLECTION

Consists of a new collection process and the assembly of another dataset;

3.2 TEST

With the new dataset generated from the recollection test if the model with the best result is working well with the new set;

3.3 PROJECT DELIVERY

Observe all the points, define what will be the treatment pattern for the images, which model behaved better and move on to the practical test phase of the project.

After training the neural network, it would be necessary to place these cameras in sub-aquatic devices so that it can start capturing images while the Artificial Intelligence identifies some of this debris, this informing which places have the highest concentration of garbage, sending messages to some central that informs the groups of divers that are focused on cleaning the beaches, which location has the greatest concentration of debris. Optimizing and facilitating the work.

Space Agency Data

Using the data provided by NASA was very necessary, as we were able to see the scale of the problem globally. Through satellite images we can see how the movement of ocean currents manages to spread the debris across the ocean. This headline "Scientists Use NASA Satellite Data to Track Ocean Microplastics From Space" taken from one of the sources provided by the NASA website, is very interesting and innovative, as microplastics are quite "hard" to find,

but through satellite images they can identify and map these microplastics. These are extremely important data, as we are now heading towards another technological revolution, the implementation of artificial intelligence that has been used over the years, but now should be used more as we have this data to help in the creation of our neural network .

Some of the NASA sources used for study:

https://podaac.jpl.nasa.gov/CYGNSS

https://www.epa.gov/trash-free-waters/toxicological-threats-plastic

https://sos.noaa.gov/catalog/datasets/marine-debris-garbage-patch-experiment-drifters-and-model/

https://www.doi.gov/ocl/marine-debris-impacts

https://www.nasa.gov/feature/esnt2021/scientists-use-nasa-satellite-data-to-track-ocean-microplastics-from-space

Hackathon Journey

It was a different experience in which it helped us to meet new people and at the same time create an idea that can help our planet. We had the opportunity to learn how the marine ecosystem is affected by the waste we produce, the choice of this topic was made in a way in which we can combine technology with the environment.

Our inspiration comes from the current situation in which we are living, the planet Earth and its overload due to pollution, many materials that over the years were discarded incorrectly and nowadays creates this race to avoid future problems with our health and well-being, in addition to influencing the planet's biodiversity. The team was initially formatted by three artificial intelligence technician students at the SENAI CIMATEC institution, and in our course we learned that A.I can help solve numerous problems, in different areas. And so when we called all the members together, we decided to work towards this idea that can help resolve a situation that is extremely urgent.

The oceans are like an underworld, like a world that hasn't been fully explored yet, some scientists say we haven't explored even 5%. And despite being so big, the consequences of pollution in the oceans are already reaching all locations, destroying the lives of those who live there. Humanity is managing to reach an advanced state of technology, ideas that were once impossible. And it is more than necessary for science itself to think of countless solutions that can solve these problems that have been created since past times. One of the challenges that were encountered was not finding a public data source that met our needs so that we could build our neural network. But before that, we managed to assemble her skeleton in a theoretical way so that one day we can put it into practice.

Our thanks go to the entire NASA SPACE team that was able to help us during the process, and to Donjorge Almeida, who had the privilege of mentoring, he helped us with ideas that could improve our project.