Relevance Detection of Ophthalmic Surgery Videos

Research Proposal: In an interdisciplinary research project, computer scientists and physicians collaborate to develop and evaluate methods for automatic detection of relevant temporal segments in ophthalmic surgery1 videos (OSVs). The main objective is the creation of relevance models allowing to detect video segments that are relevant for educational, scientific, or documentary purposes in medicine. Relevance models are produced by machine learning algorithms, which are trained using OSVs that were annotated by surgeons. Important instances of relevant OSV segments are irregular operation (OP) phases, which deviate from the usual procedure used in quasi-standardized ophthalmic surgeries. The automatic detection and classification of irregularities that occur more frequently and hence allow training of machine learning algorithms, provide an additional benefit for the creation of OSV datasets targeted at medical education or research. The development and evaluation of automatic classifiers of irregularities therefore represents another research objective of this project. Relevance models can be used to compress and store OSVs efficiently. This project will develop and evaluate appropriate methods and algorithms to achieve this goal. Finally, we want to demonstrate that relevant OSV segments are useful for medical research by addressing three specific medical research questions related to a certain type of ophthalmic surgery (cataract) using video analysis. Hypotheses: (1) Regular and irregular OP phases as well as common irregular events in OSVs can be detected and classified automatically by machine learning algorithms. (2) The automatic relevance prediction of video segments enables a significant increase in efficiency for storing and archiving OSVs. (3) Automatic analysis of OSVs facilitates the investigation of quantitative medical research questions related to complication rate and various OP conditions. Methods: Classifiers and compression algorithms are developed using an iterative process comprising design, evaluation and improvement. Quantitative medical research questions are addressed by statistical correlation and variance analysis. Innovation: Currently, automatic analysis of OSVs is a young research field with a small number of publications (mainly concerning phase detection in cataract surgeries). Relevance modelling of OSV segments is a new idea, as is relevance-driven compression of OSVs. Moreover, the use of video analysis to investigate quantitative research questions related to ophthalmic surgeries represents a new approach in medical research. 1 Ophthalmic surgery is concerned with the human eye.

Computer scientists and physicians have collaborated in a multidisciplinary research project with a focus on computer science to develop and evaluate methods for the automatic detection of relevant temporal and spatial content in ophthalmic surgery videos. The main goal of the project OVID ("Relevance Detection in Ophthalmic Surgery Videos") was to model relevance with respect to the use of video segments for medical teaching, research and documentation. For this purpose, mainly artificial intelligence (AI) methods were used and various neural network models were developed and applied for content recognition. This allowed relevance models to be learned automatically by machine learning methods, using surgical videos annotated by surgeons as training data. On the one hand, these relevance models can be used for efficient compression and storage of surgical videos, and on the other hand, they can also be used for automatic recognition of a wide variety of relevant content (e.g. surgical phases, instruments, iris, pupil, the lens used in cataracts, etc.). This makes it possible to capture important sections in an eye operation fully automatically, index them and make them usable for subsequent content-based searches. These models also allow large forensic studies of specific complications over a large number of operations or patients. In the OVID project, it was successfully shown that specific problem situations or complications in videos of cataract operations can be found through the targeted use of artificial intelligence and algorithmic methods of computer science. For example, the occurrence of pupil reactions (unpredictable enlargement/reduction of the pupil, which is accompanied by a dangerous change in the anterior chamber depth) was automatically detected. Furthermore, object detection and tracking in the video made it possible to measure the unfolding time of artificial lenses as well as their stability. The results of the research project have been published in several papers at high-level conferences on medical image analysis, pattern recognition, and multimedia retrieval.