Adding Audio-Visual Cues to Signs and Symptoms for Triaging Suspected or Diagnosed COVID-19 Patients

The COVID-19 pandemic has placed unprecedented stress on hospital capacity. Increased emergency department (ED) patient volumes and admission rates have led to a scarcity in beds and the need to construct field hospitals in some regions. Bed-sparing protocols that identify COVID-19 patients stable for discharge from the ED or early hospital discharge have proven elusive given this population's propensity to rapidly deteriorate up to one week after illness onset. Consequently, a significant number of stable patients are unnecessarily admitted to the hospital, while some discharged patients decompensate at home and subsequently require emergency transport to the ED. In order to conserve hospital beds, there is an urgent need for improved methods for assessing clinical stability of COVID-19 patients. Our project goal is to develop audiovisual tools to predict cardiopulmonary decompensation from facial videos captured from the ED and at home via telemedicine platforms. We will use explainable artificial intelligence (AI) and machine learning (ML) algorithms to derive criteria for predicting impending deterioration from health-relevant audiovisual features and provide explanations in terms of the clinical details within the electronic medical record. Successful completion of this project will provide the groundwork for prospective evaluation of these tools in a COVID-19 patient population. Once validated, these tools will augment provider clinical assessments of COVID-19 patients both at the bedside and across telemedicine platforms during virtual follow-up. More broadly, the techniques and algorithms developed in this project are likely to be applicable to other high-risk patient populations and emerging platforms such as telemedicine.