High-Level Project Summary
We are making an application which will allow users to perform Key Tasks on Computer like mouse control, keyboard, etc through Hand gestures & Eye Gestures. The use of mouse and keyboard can be completely avoided by adapting gesture control technology, thus being cost effective and reducing electronic waste. Most importantly by providing contactless operations this technology avails users to avoid touch, which ultimately lessens the spread of virus. On the ground of Social cause, this technology can be beneficial in multiple sectors. The government can implement this technology in fields of Aviation as well as ministry of Electronics & Information Technology and for Manufacturing too.
Link to Project "Demo"
Link to Final Project
Detailed Project Description
Introduction
The proposed topic "Eye Sight" is developed in order to make human computer interactions easier using gestures recognition technology. The use of a physical controller like mouse, keyboard for computer operations hinders natural interface as there is a strong barrier between the user and computer. Interaction between humans comes from different sensory modes like gesture, speech, facial and body expressions .In this project we have developed a gesture recognition system which can efficiently track Hand, Voice and Eye Gestures. Our system translates the detected gesture into actions such as virtual painting, zoom in/out operations, volume increase/decrease, right and left click mouse controls, virtual keyboard, emailing and all general computer operations. The software can be useful for vide range of users. It can be used by health care departments, schools and students in online teaching, can be used by gamers for realistic and immersive experience, most importantly it can be used by disabled users. Our results show that an intuitive HCI can be achieved with minimum cost requirements.
Coding languages : Python, React js, javascript, CSS.
Hand Gestures Recognition
This feature facilitates the user with contactless controlling system. The user can perform all the basic functions of the computer without touching the mouse or keyboard. The user can move the cursor with his hands without even using the mouse.
Eye Gesture Recognition
This gesture recognition system uses gestures captured by eye movements of the user. The users can use virtual keyboard with their eye movements. Users need not click any buttons on keyboard and can still type anything with eye gestures. The user can also detect the position of the cursor using this technology.
Voice Control Recognition
Voice control recognition detects the voice commands given by the users and performs operations. A user can even send an email using voice commands. This technology can come in handy when a user can not physically touch the system devices.
Primarily, our application will be developed as a Web Extension. Further, we will be developing desktop based application
for Windows, Linux & Mac OS.
Space Agency Data
https://nssdc.gsfc.nasa.gov/
https://data.nasa.gov/
https://science.nasa.gov/national-space-science-data-center/
Hackathon Journey
To begin with, we as a team are extremely overwhelmed and highly grateful for being able to be a part of the Space Apps Challenge. Having the opportunity to pitch our idea on such a huge platform was one of the finest experience. The world being hit with a global pandemic, we thought of coming up with a contactless technology, that can be used by everyone and in all sectors avoiding the physical touch of any surface. Making this an entirely touch free experience. The greatest lesson that we learned from this challenge was to believe in team spirit, working with a team can get tough with lack of co-ordination but we realized that teams can make the impossible happen. We definitely had the first hand experience of "One for all and all for one". We discovered a lot through out the journey, hence we are immensely thankful.
References
Tool : AI, Deep Learning, Computer Vision, Neural networks, Machine Learning, opencv, pytesseract, mediapipe
https://youtu.be/8gPONnGIPgw
https://youtu.be/9iEPzbG-xLE
https://youtu.be/jzXZVFqEE2I
https://youtu.be/6400ShqS9BY
Tags
#AI, #ML, #ComputerVision, #DeepLearning, #Automation, #Innovation,