High-Level Project Summary
PREVAIL is a real-time collaborative application for mission activity logging. It allows the Mission Control Center flight operations team and any authorized participant to create, edit and/or view collaborative mission logs. Team Spatiam’s PREVAIL is unique in supporting Extravehicular Activities (EVA) by using artificial intelligence to help enhance logging with critical automation and mission data, automatically including CAPCOM transcripts, location and sensor metadata, assisting in the decision making and post-mission analysis. PREVAIL is designed considering the communication LunaNet will provide on the Moon. PREVAIL has been prototyped successfully!
Link to Project "Demo"
Link to Final Project
Detailed Project Description
What is PREVAIL?
PREVAIL - Primary EVA Intelligent Logging is a real-time collaborative application for mission logging. It allows Mission Control Flight Operations Teams to create context-rich and actionable EVA logs that can be shared real-time with other team members and the scientific community.
With PREVAIL, any mission control operations team can:
- Create mission activity logs: Per each mission and extravehicular activity, there is a master log which incorporates all entries for all operations team members. A common timeline and metadata help to correlate events and entries among the team roles.
- Augment log entries with EVA data: Automatic CAPCOM speech transcriptions, spacesuit Primary Life Support Systems sensor data, location data, and relative among others augment logs with the information available at the time.
- View, real-time, the log entries of all Flight/EVA operations members, as desired.
- Filter mission activity logs content by author, team/role, timestamps or content.
- Use Define Role-based policies to prevent unwanted viewing, editing or deletion of log entries.
- Publish logs to other non-operational teams as desired.
What are the benefits of using PREVAIL?
Compared to existing logging tools used, PREVAIL offers the following key benefits:
- Real-time collaboration, allowing flight/EVA operations users to create, edit and view common Mission Activity logs, compared to existing isolated logs which are shared at the end of each shift. This accelerates convergence of analysis and decision making and reduces potential errors introduced by delays.
- Augmented analysis capabilities via artificial intelligence, allowing users to get AI-based assistance in log and EVA data analysis. Currently it is up to each human to transcribe and view/detect what’s happening.
- Better information logging with EVA Data Synchronization, allowing users to sync, match log entries with relevant EVA data. This contributes to
- A great user interface, with high level of data visualization, filtering and engagement.
- Privacy and Security, with role based access control and authentication.
- Designed taken into account EVA communications through LunaNet.
PREVAIL in detail
PREVAIL is a web-based application that can be accessed from any computer. Users with a personal computer with an Internet Browser can experience it. Following is the key functionality of PREVAIL.
Onboarding and Login
At time of enrolling (during mission preparation), each user is onboarded by the administrator based on their role, e.g. Operations FLIGHT, SURGEON, AFLIGHT, CAPCOM, EXT-Scientist, etc. This would define the access and privileges in the system, i.e. whether users can create or view real-time mission activity logs.
Users would then login to the application using their Single Sign-On credentials e.g. NASA Active Directory.
Note: At this stage of the prototype implementation, login has been implemented with a username and password combination.
Figure. PREVAIL Login screen
Mission Logs Dashboard
After login, users will be presented with the mission logs they are authorized to see/edit. This dashboard shows logs for each mission, its author, users associated with the logs, and metadata about it such as time created, last modified, tags and additional notes. There are filter options to view the logs per author, role, date and an advanced filter for more complex queries.
From the dashboard authorized users will be able to create new mission logs or open an existing mission log to create new log entries.
Note: We envisage each log will be processed by Machine Learning Operators to automatically create tags and labels (e.g. safety, navigation) in addition to the ones manual created. This is not implemented on the prototype due to time constraints.
Figure. PREVAIL Mission Logs Dashboard
Mission Activity Log Screen
Once creating or opening a Mission Activity Log, the specific activity log entries are shown. In this, a real-time collaborative log allows flight operations (and any authorized viewer) to view and/or add entries. The Activity Log Screen allows users to:
- See a timeline view of events for the entire activity including all Flight/EVA Operations participants.
- Filter the activity log entries per author (e.g. only seeing my own entries), per role, per date/time, and advanced filtering per keywords, tags and other metadata.
- See the following information per entry: Date of event, Author/Role, Content: Activity and File, Tags - manually input (and auto-generated via Machine Learning literal operators in future). Note that in the case of image files, these are processed by a computer vision algorithm to automatically extract labels (view architecture section later in this document).
- See EVA data: This is critical EVA data added at the moment of entry. This is automatically syndicated via PREVAIL’s data sync (view architecture section later in this document).
- Primary Life Support Systems sensor data: Heart Rate, Temperature, spacesuit pressure, and Selenographic coordinates. Selenographic coordinates are processed via Google maps and NASA extensions to show a map of its location on the Moon.
- CAPCOM transcript: CAPCOM transcript are automatically included in the log entry, using PREVAIL’s data sync and Artificial Intelligence services (view architecture section later in this document).
- See Additional Details: Date of creating, last modified, Users, Type of file, etc.
- And, to Add new entries in the log.
Figure. PREVAIL Mission Activity Logs details
Mission Activity NEW Entry Log Screen
Adding a new entry is straightforward. Press the Add new entry button or (future) execute the keystroke combination to open the NEW Entry Log Screen. In this screen the user can create a new Log Entry.
- Mission, Activity, Role and User are auto populated.
- The user has the option to make the entry private (it is shared by default).
- The user adds the log text and it has the option to add a file as well.
- The user can Add EVA Data to it (relevant in many situations in which the user would like to remember the data as it was seen at the time of logging). Adding the EVA data is just a click-away as PREVAIL Data Sync syndicates this data and makes it available to the user. The data includes CAPCOM transcript and Primary Life Support System metrics such as Astronaut heart rate, external temperature, Spacesuit pressure, and selenographic coordinates.
Figure. PREVAIL Mission Activity New Entry Log Screen
PREVAIL Architecture
PREVAIL is a web-based application that can be accessed from any computer, and it can be deployed in cloud or on-premise as a virtual server application. Users need a personal computer with an Internet Browser to experience its functionality. For its prototype, the architecture has been fully implemented in a Virtual Machine in Amazon Web Services computing (EC2) instance.
Following are the key architectural elements of PREVAIL.
Figure. PREVAIL System Architecture. Note this architecture has been fully prototyped!.
- End-user portal (aka Front-end services): The front-end services deliver the end-user portal. It contains the UI elements, web resources and services creating a real-time, engaging interface for users to create and visualize large amounts of data. Real-time operations are achieved by maintaining a socket between the front and backend to exchange updates. The front-end services are built using Angular framework for client applications.
- Backend web services: The Backend web services contain the application logic, authentication and role/authorization services, and the backend API. The Application Programmable Interface (API) is the main interface between the different end-user/data interfaces and the application logic and database. The backend web services have been built using the django Python web services framework.
- Artificial Intelligence Services: The PREVAIL Artificial Intelligence (AI) Services enhance the data logging activity by transforming, labeling and visualizing data logged by the system. Current services include automatic transcription of CAPCOM communications, automatic labeling of uploaded data (currently images, in future text, video and other types), and Geographic visualization mapping (selenographic coordinates to Moon surface map). The artificial intelligence services used in PREVAIL are cloud API-services (Google Speech-to-text, Google Cloud Vision, and Google Moon Map APIs with NASA extensions).
- EVA Data Sync: EVA Data Sync services allow for data syndication to augment the data logging capabilities of the systems with EVA data. We believe this is critical to ensure sync between the log entries (text/files) by the flight operators and data seen at that specific moment. The Data Sync services are currently prototyped together with an emulation of the Astronaut CAPCOM and Primary Life Support Systems (see prototype description below).
- Database: The Database stores PREVAIL System Data, i.e. logs, its associated metadata, user/role information among others. It has been implemented in a relational QSLite3 Database.
PREVAIL EVA Data Sync Prototype
In order to properly test PREVAIL and its user experience, we emulated the Astronaut (Commander or CMR in picture below) CAPCOM and key Life Support System Hardware functionality to connect it to PREVAIL EVA Data Sync. Note the EVA Data Sync is pre-configured to map this specific Astronaut data to a mission and activity.
- We used a Raspberry Pi with a microphone to emulate the CAPCOM radio and the Life Support System Sensors (at this time we emulated the sensors generating sample data).
- We created a software program (Python) to detect speech (sentences), then collect its timestamp and send the audio segment for CAPCOM transcriptions (using Google text-to-speech).
- The transcription results are packed together with the available sensor data in the payload of the API call to create a log entry (CAPCOM transcription) and have the EVA data available in all logs for that activity.
This way users of PREVAIL can automatically add key data to the log as seen at that moment, reducing time (and associated errors) correlating data on distinct systems.
Figure. PREVAIL EVA Data Sync (and Astronaut CAPCOM/PLSS) prototype details.
Space Agency Data
We used the following Data, Software and Tools (the reference section contains the links):
- Several flight journals and logs, included Apollo 11, Apollo 13 and Apollo 15. We learned about the processes and typical examples of communications and logging. Of special value, the Apollo 15 Flight Journal included several EVA events which helped us to draw examples.
- NASA Moon in Google Earth. Google Moon maps augmented with NASA resources. We use it to automatically map incoming EVA selenographic coordinates to a position, which can be added automatically into the log as EVA data.
- Description of NASA Flight Operations, describing personas and their specific responsibilities.
- Several NASA documents related to spacesuits and Primary Life Support Systems that helped us to create a working HW prototype to simulate CAPCOM radio and PLSS sensor data.
- Resource data of this challenge, we used it to gather requirements and understand how several features would be used (e.g. filtering logs per different attributes).
Hackathon Journey
Our inspiration can from our enthusiasm about space, and the challenge of creating a modern real-time collaborative application for NASA.
We followed a typical rapid prototyping approach:
(a) We review the challenge and created a master plan.
(b) We assigned roles and responsibilities
- Heather - Research, instructional design and UX
- Nayla - UX and UI
- Shawn - Front-end development
- Alberto Jr. - Back-end development
- Eric - Data Sync and spacesuit CAPCOM and PLSS HW emulation
- Alberto Sr. Research, Requirements, coordination and documentation.
(c) We established ceremonies, with stand ups approx. every 2.5 hours.
(d) We helped each other.
We made great analysis and prototyping:
Persona analysis (example)
Design System
Prototype UI
Prototype transcriptions
We progressed the system prototype
At this stage of the prototype implementation, we have made great strides into creating the final project we envisioned.
The backend API has been fully implemented to support the addition and creation of mission logs and log entries, as well as multi-user support and sharing capabilities of logs.
Figure. Example API call to add a log to the mission activity.
Our implementation fell short on the frontend side, where a limited client has access to a small subset of the operations the API allows.
However, having the API and frontend mockups done, we believe Prevail is currently very close to it's completion!
References
- NASA. Apollo 15 Flight Journal. Preparations for EVA-1. Available at https://www.hq.nasa.gov/alsj/a15/a15.eva1prep.html
- NASA/TP–2017–219457. Operational Assessment of Apollo Lunar Surface Extravehicular Activity.July 2017. Available at https://ntrs.nasa.gov/api/citations/20170007261/downloads/20170007261.pdf
- Deans et al. NASA Eames Research Center and others. Minerva: User-Centered Science Operations Software Capability for Future Human Exploration. 2017 IEEE Aerospace Conference. Available at https://ntrs.nasa.gov/api/citations/20170002363/downloads/20170002363.pdf
- NASA.com. Spacewalk support. Available at https://www.nasa.gov/audience/foreducators/k-4/features/F_Space_Walk_Support.html
- NASA Johnson Space Center. The space shuttle and its operations. Extravehicular Activity Operations and Enhancements. Page 110-129. Available at https://www.nasa.gov/centers/johnson/pdf/584725main_Wings-ch3d-pgs110-129.pdf
- NASA. Artemis EVA Flight Operations. Preparing for Lunar EVA Training and Execution. February 2020. Available at https://www.nasa.gov/sites/default/files/atoms/files/topic_4-_preparing_for_lunar_training_and_execution.pdf
- Wikipedia. Flight Controller. https://en.wikipedia.org/wiki/Flight_controller
- Wikipedia. Spacesuit. https://en.wikipedia.org/wiki/Space_suit
- Primary Life Support System. Wikipedia. https://en.wikipedia.org/wiki/Primary_life_support_system
- Ars Technica. Apollo Flight Controller 101: every console explained. Available at https://arstechnica.com/science/2019/12/apollo-flight-controller-101-every-console-explained/
- NASA. LunaNet: A Flexible and Extensible Lunar exploration communications and navigation infrastructure. Available at https://esc.gsfc.nasa.gov/news/_LunaNetConcept
- NASA. Moon in Google Earth. Planetary Content. Available at https://ti.arc.nasa.gov/tech/asr/groups/intelligent-robotics/planetary/moon/
Tags
#prevail #log #real-time #collaboration #operations #eva #logging #artificial-intelligence #ai #plss #spacesuit #location #nasa #moon #lunar #Artemis #hardware #capcom #flight #spacesuit #extravehicular #activity
Global Judging
This project has been submitted for consideration during the Judging process.