Semantic Processing of Security Event Streams (SEPSES)

As complex IT systems are becoming more and more deeply ingrained in our everyday lives, their integrity and operation is threated by increasingly sophisticated attacks. Relatively simple user activities (such as logging into an online banking system, using a communication service etc.) often require a complex series of steps that provide a surface for attacks. Therefore, events are usually documented and, along with general information about the system state, stored in extensive log files. The overwhelming volume of these disparate logs, which need to be monitored constantly, poses a major challenge for security analysts. At present, these logs are often scattered across systems and are not comprehensible for computers. Therefore, monitoring them for clues of attacks cannot easily be automated. In large systems with many users, where manual monitoring is infeasible, many security incidents are therefore only identified with considerable delay or, even worse, not at all. The project SEPSES takes an innovative approach to tackle these challenges. It enables computers to integrate and interpret streams of log information from many sources in real time. This allows computers to reason about potential malicious activities and support security analysts in identifying, tracing, and eliminating threats in a timely manner. To this end, SEPSES unifies theories from security research, data stream processing technologies, and methods developed in the Linked Data and Semantic Web research communities. To facilitate continuous real time monitoring of complex event streams, a set of conceptual and technical challenges need to be overcome. First, large-volume data streams must be combined and converted into a machine-understandable representation. This requires modeling security knowledge in a way that can be interpreted by computers. Second, event streams need to be integrated and consolidated at a central location. This will facilitate context-rich methods that go beyond searching for individual events. Hence, it will become possible to link individual isolated events and conduct context-aware analyses. Overall, this will result in a comprehensive view on system activities, allow security analysts to derive explanations for observed behaviors, and make it possible to identify attack patterns automatically. Methods developed in the course of SEPSES provide a foundation for learning of new attack patterns and the continuous exchange of dynamic security knowledge. This could, for instance, result in a public collection of known attack patterns in the Linked Open Data Cloud that anyone can subscribe to. Moreover, the developed methods form the basis for advanced diagnostic methods and will provide a platform for innovative security applications such as comprehensible visualization of sequences of malicious events, semantic forensic analyses, and structurally rich log data mining.

The digitalisation of all aspects of our lives comes with a critical dependence on information technologies, which are facing increasingly sophisticated attacks that threaten their confidentiality, integrity, and availability. To detect and respond to such attacks, it is necessary to collect and analyze information about activities - which are typically stored in log files. These files may provide clues about malicious activities, but the overwhelming volume of these logs poses a major challenge for security researchers and analysts. At present, such textual logs come in various formats and are scattered across systems. Therefore, monitoring them for clues of attacks cannot easily be automated and thus, many security incidents are only identified with considerable delay, if at all. To tackle these challenges, the research project SEPSES aimed to automatically transform log data from disparate sources into a knowledge graph and to enrich this graph with information about threats, weaknesses, vulnerabilities, and countermeasures. Once log data has been extracted and transformed this way, it becomes possible to establish links between seemingly unrelated events - such as sequences of steps in an attack - and to search for patterns that provide clues about malicious activities. This new approach to analyze log data aimed to improve detection, understanding, and response to threats. To achieve this goal, SEPSES combined security research with semantic technologies and knowledge-based approaches in order to develop techniques to harmonize and integrate large, heterogeneous log data into a machine-interpretable representation. In order to provide an integrated perspective, it was necessary to develop a formal description of the events typically covered in log files. This made it possible to search for generic patterns of malicious activities, not only in local log archives, but also across machines and platforms in distributed environments, for which we developed a virtual knowledge graph approach. The concepts, methods, and resources that result from the project provide a foundation for the development of comprehensive tooling for semantic security analyses. Through integration, automation, contextualization, and automated machine-interpretation, such tooling can contribute towards increased situational awareness, reduced alert fatigue, and fast response times in the future. The developed methods also support the exchange of security knowledge, provide new concepts for the discovery of novel attack patterns, and create a foundation for further security research, e.g., into graph-based machine learning techniques to analyze log data.