CRDP Lebanese Space Phoenix

https://drive.google.com/file/d/10I1q2ePX2MZYQzknTwi1vpgcQBlXSUa2/view?usp=sharing

https://drive.google.com/drive/u/1/folders/1tV-JaeFUtJAEF3R7wvhTUBkbLDoVKt11

Project Summary

UrbanWatch is a web based application that provides location-based data-oriented recommendations for governmental organizations (such as municipalities) and stakeholders with the aim of guiding the users towards more durable and resilient future urban plans.

In order to show the efficiency and importance of this application, hybrid data (earth observation satellites (EOS) data, unmanned aerial vehicles (UAV) data, and in situ rovers’ data) was collected for a local region in Lebanon: The Bekaa Valley. Data was combined and processed to give recommendations based on location. 

Although UrbanWatch came as a response to the Nasa Space Apps Challenge (https://2021.spaceappschallenge.org/locations/beirut/), it is a vital application for Lebanon’s future and urban development. 

Listed below are some of the issues that UrbanWatch can help in either solving or reducing:

• Lebanon lacks sufficient data on alternative energy resources that can be very essential in the battle against pollution and dependence on nonrenewable polluting resources such as fuel. The data gathering, merging, feature extraction and recommendations found in UrbanWatch can contribute by advising users on the proper harnessing of such resources where available.
• Lebanon witnessed in the last decade many sporadic fires that affected the green areas. UrbanWatch can help local municipalities and citizens build a risk-free green environment. To achieve environmental sustainability and contribute to the national environment growth, recommendations to increase green areas, building risk free buildings for the citizens, and achieve better agricultural patterns, are all based on output data proposed by the application. 
• There are very rare studies at the local level in Lebanon that involve real-time and recent actual data. Such data permits collaboration of various sectors (construction, urban development, legal, environmental, among others) to implement the appropriate administrative rules and laws for different scenarios based on region specific data.  

Methodology

With the aim of providing the most accurate recommendations and estimations to the user, our web application works on combining the power of satellite and Earth Observation techniques, which provide large scale data and estimations both spatially and temporally, with drone and earth-bound techniques, which allow for higher refinement and granularity of the data, especially at the spatial level. For example, a considerable number of satellite data resources available freely online provide data at a maximal spatial refinement level of 5 to 10 km radius. On the other hand, drone and earth-bound data sources can allow much higher refinement up to a spatial point: as such, we can retrieve data for a single spatial point defined by its coordinates.

An important challenge which poses itself is then the methodology of combination of the data arising from these multiple sources. Data can be combined on multiple dimensions: namely spatial, temporal or (optimally) spatio-temporal. Currently, as we do not have drone data available over long periods of time, we decided to combine the data only over the spatial dimension. As such, for a given location, we find the NASA-provided data relevant to it, and the drone data we have for this location, and combine them together (regardless of the time differences between the two data points, for now).

As our drone data are spatial points, and the NASA-retrieved data are polygons with lower resolution, the data combination technique consists of applying a “point-in-polygon” approach which finds, for a given spatial point (drone) the polygon to which it belongs (NASA). From here, the variables can be fused together and aggregated over the spatial field if needed using multiple possible techniques, the simplest being an average over the spatial field. A detailed (mathematical) definition of this approach is provided in the following:

The “point- in- polygon” test is famously used in computational geometry. Given a point R and an arbitrary closed polygon P represented as an array of n points: P0, P1, ... , Pn-2, Pn-1where P0 =Pn . How can I tell if R is inside P? There are two main approaches: even-odd or parity rule and winding numbers. Even-odd : draw a line from R to S that is certainly outside P. If RS crosses the edges ei =(Pi,Pi+1) odd numbers of times, the point is inside P. Winding numbers: based on the winding number of R with respect to P, which is the number of revolutions made around that part while traveling once around P.

Using image processing techniques, UrbanWatch also allows to extract features that may be relevant to certain images taken via drones. For now, we implement a simple approach to extract the vegetation ratio from retrieved images. The simple approach can be seen in the below schema. Surely more advanced techniques can be applied to obtain more relevant, accurate and real estimations of the green vegetation ratio, such as machine learning segmentation techniques or even neural networks adapted to green segmentations. However, for the purpose of this short hackathon, we have adopted this simple approach; and more advanced approaches will be developed later on.

Variables Extracted and Processed from Different Sources:

In this project, we studied 8 different variables: air humidity, air speed, atmospheric pressure, carbon monoxide, soil humidity, temperature, NASA Terra Vegetation Continuous Fields (VCF5KYR) and Vegetation Ratio (drone).

Due to time constraints, we were only able to give recommendations for two studied variables: carbon monoxide and vegetation ratio. As such, this implementation aims at showing the potential of our application and is by no means a final version.

For the carbon monoxide variable, we took the ratio RS/R0 , where RS is the internal resistance of the sensor which changes by gas concentration and R0 is the sensor resistance in 1000 ppm concentration of LPG. If the calculated ratio is greater than 4.4, the air quality is appropriate for vegetation, seed germination and living, else there are dangerous levels of carbon monoxide. In the latter, it is better to report to the authorities. 

As for the vegetation ratio, it is the average of the vegetation ratio obtained from the drones and the satellites. For the meantime we considered the average of the two values, but this is not completely true; better schemes of interpolation can be adapted, accounting for the spatial and temporal autocorrelation among the observations. If the average vegetation ratio is greater than 0.45, there is a healthy level of vegetation, else, levels of greenery must be reconsidered (add more trees, plants, vegetables …).

Case Study: 

We gathered data from the Bekaa Valley, Lebanon at different points above the Litani River. The data was gathered through a drone, equipped with a camera and multiple sensors: Gas Sensor (Carbon Monoxide), Atmospheric pressure and temperature sensor, Photoresisitor sensor, Soil and Air Humidity Sensor, and an Anemometer. Note that multiple visits were made to the same area for around two weeks to collect data. 

The Phoenix sensor equipped drone used for data extraction:

A sensor equipped plane used for data extraction:

The rover for land-sample data extraction:

Front page of the application

User selected region

Post Processing and Results Page with Recommendations

App Development: 

First of all, our web app is divided into 2 parts, the front-end and the back-end. The front-end consists of the user-interface, where the user visualizes and interacts with the application. The back-end consists of all the data processing for the application. The front-end and back-end have to be connected to each other in order to have our app fully functioning. The front-end includes an integrated map of the earth that displays a satellite view. From this satellite view we offer the user the ability to select a region of his/her choice. The selected region is considered to be in the form of a polygon. After the selection of this region, the coordinates are then sent to the back-end. 

In the back-end, the coordinates retrieved from the front-end are then unpacked and intersected with the entries in the database along the spatial dimension. Given the coordinates, we apply a point-in-polygon approach intersecting the vertices of the chosen polygon with the polygons in the database. If data is found, then the relevant records are returned in JSON format to be displayed on the front-end. 

 Flow of the application: 

• The user signs in to the application with the given username and password.
• The user then is taken to a page where a map will appear with the ability to locate the region he/she desires. 
• Then there is a tiny panel on the top right corner that consists of two buttons (Draw polygon, and Delete Polygon).
• Once the user locates the region he/she wants to select, the user then clicks on the draw polygon button and starts to draw around the region they desire by selecting certain points. 
• Once the user draws over the region they want, the user presses on the draw polygon button again, or presses the Enter button on the keyboard. 
• Once the user is sure that this is the region they would like to process, they will click on the “Process Region” button in the top-left corner. 
• The user will then be navigated to a page where there will be a full view of recommendations for the selected area including all the major aspects and attributes needed to be taken into consideration when dealing with urban development. These recommendations will be displayed based on the data processing in the backend of the application. 
• The user can constantly check for new recommendations by sign in whenever needed. 

At this level the application was developed with the aim of showing the potential of fusion of satellite data obtained from space agencies sources, such as NASA's, with locally made and manipulated UVs. Due to time limits, the application at this stage only focused on one satellite dataset obtained from NASA, namely the "VCF5KYR v001: MEaSUREs Vegetation Continuous Fields (VCF) Yearly Global 0.05 Deg" dataset. This dataset provides data in three bands: Percent of Tree Cover, Non-Tree Vegetation, and Bare Ground. These values are obtained through a bagged linear model applied on " Long Term Data Record Version 4 (LTDR V4) data compiled from Advanced Very High Resolution Radiometer (AVHRR) observations".

For our use case, we focused on the second band in the provided data, namely "Percent of Non-Tree Vegetation". To extract this data, which can be downloaded in GeoTIFF format, we used the open source software QGIS as an layer on top of GDAL in order to apply a polygonization algorithm allowing us to produce a dataset from the GeoTIFF data, after subsetting on the second layer only. Each obtained polygon will be defined by its vertices (with their coordinates) and can thus be easily manipulated in Python using Geopandas, a library we are quite familiar with. This satellite data is then combined with the drone data using the approached detailed in the "Detail Project Description" section,

The usage of this data shows the possibility of applying this or similar approaches to integrate other datasets locally. Indeed other datasets have been consulted and downloaded, but unfortunately were not applied in the current application due to the need of prioritising certain implementations over others. Among the consulted datasets we cite the "Future Development Threat from Agricultural Expansion" , the "Global 3-Year Running Mean Ground-Level Nitrogen Dioxide (NO2) Grids from GOME, SCIAMACHY and GOME-2" and others.

The VCF5KYR dataset can be accessed here: https://lpdaac.usgs.gov/products/vcf5kyrv001/

The other mentioned datasets can be accessed respectively here:

http://s3.amazonaws.com/DevByDesign-Web/MappingAppsVer2/DevRisk/index.html

and here: http://sedac.ciesin.columbia.edu/data/set/sdei-global-3-year-running-mean-no2-gome-sciamachy-gome2-1996-2012/data-download

Our journey for the hackathon started one month ago when we were asked to participate in the Nasa Space App Challenge. The team started to build up gradually based on enthusiasm and expertise. The opportunity was great, for after all the dark days that our beloved country passed though and still is, a light appeared for us all. We chose our challenge based on our country’s need for a solution to its accumulating problems among which are pollution, destroyed areas due to recurring fires, and unorganized urban laws and guidelines. Our team has a very enthusiastic member who worked previously on a Flying Telescope, which also gave us motivation to go ahead with the challenge.

We started this adventure by reading many research papers on hybrid data fusion and combination. Data collection started in Bekaa Valley, Lebanon, using a drone and a rover equipped with sensors and cameras . Satellite data was extracted from Nasa sites, downloaded and decompressed for use in the application.

Our team faced many setbacks such as power outage, internet disconnection , and transportation issues due to fuel shortage. We were lucky that two of our team members lived outside Lebanon and helped in downloading media files and satellite data. The university of USEK also helped us in providing rooms with constant power supply and internet connection throughout the hackathon.

Our team would like to thank Mr Antoine Tannous, our local lead for Lebanon, for his excellent leadership, patience, and kindness, Mrs. Houda El Khoury, the Head of the IT department in the CRDP for believing in us and supporting us throughout this journey, Dr. Blanche Abi Assaf, Head of education department in CRDP, Mr. George Nohra, Head of CRDP, and all the wonderful volunteers for their extensive and endless support.

Front End:

• JavaScript
• React.js 
• Mapbox GL JS

Back End:

• Python
• Flask

Database Preparation & Feature Extraction:

• OpenCV
• Geopandas
• Pandas
• QGIS
• GDAL

Programming language used to extract data from the drones: 

• Arduino 

Cameras used:

NCTS ACTION CAMERA 4K NCTS-X7W

Sensors used:

1. MQ9 Gas Sensor
2. KY-018 Photoresistor sensor
3. BMP180 Pressure and Temperature Sensor

References:

https://www.euro.who.int/__data/assets/pdf_file/0020/123059/AQG2ndEd_5_5carbonmonoxide.PDF

(normal levels of CO from WHO)

https://dx.doi.org/10.3389%2Ffpls.2016.00572

(CO at higher concentrations but still in the normal range can be beneficial)

https://www.abe.iastate.edu/extension-and-outreach/carbon-monoxide-concentrations-table-aen-172/

(CO levels charts)

VCF5KYR_User_Guide_v1.pdf

(Vegetation Continuous Fields)

https://doi.org/10.1016/S0925-7721(01)00012-8

Point in Polygon Test

Data sources from space agencies: 

https://lpdaac.usgs.gov/products/vcf5kyrv001/

MEaSUREs Vegetation Continuous Fields (VCF) Yearly Global 0.05 Deg

#drone #sensor #spatial_geometry #satellite #UAV #rover #urbanism #pollution #development #image_processing #agriculture #React.js #green_ratio #application #Mapbox_GL_JS #Python #Flask #maps #coding #arduino #

This project has been submitted for consideration during the Judging process.