Uncovering Clinical Evidence in COVID-19 Publications: An Integrated Search via Text & Images

Project Summary: Uncovering clinical evidence in COVID19 publications: An integrated search via text & imagesThe proposed research aims to develop and advance tools for using image-data appearing in scientificpublications, in addition to text, in order to expedite effective access to COVID-19 published information.Current efforts aiming to address the COVID-19 pandemic include devising treatment, understanding virusmechanisms, detecting infection and antibodies, and ultimately - developing a vaccine.All these efforts require effective access to biomedical information related to the virus. The Allen Institute hasrecently released the CORD-19 dataset - a large, continually updated collection of scientific literature pertainingto COVID-19 and Corona viruses. This dataset comprises tens of thousands full text articles, forming a basis fortext-mining tools that will support access to information pertaining to COVID-19.Notably, much of the evidence within these publications is provided in the form of figures. Furthermore, regionswhere such evidential images occur are rich in information.While biomedical text-based mining tools are being quickly developed and offered for accessing this dataset,images, which contain key clinical and biological information, are not considered. Even outside the COVID-19realm, little has been done so far to utilize images within publications, despite the fact that they provide importantcues about the relevance of the information embedded in articles.Our premise, which is supported by our own and by other informaticians and clinicians experience, is thatinformation derived from images can (and should) be directly incorporated into the biomedical - and specificallyinto the COVID-19 - document retrieval and extraction. Doing so will improve accurate access to relevantarticles, while pin-pointing significant evidence within them, and expediting access to much-needed criticalinformation. The work on this project will result in methods and tools that take advantage of both image- andtext-data, facilitating more effective and focused retrieval and mining, thus better supporting speedy data-intensive discovery in the context of COVID-19.