Grants and Contributions:

Grant or Award spanning more than one fiscal year. (2017-2018 to 2022-2023)

Despite the advances in software development, the increasing complexity of software systems makes it difficult to produce fault free software. The residual faults often manifest themselves during system execution causing severe software failures, security breaches, and performance degradation. To diagnose the root causes of failures, software developers and system administrators frequently resort to the analysis of logs, generated during system executions. Major companies like Ericsson, Google, and Microsoft collect logs to analyze, fix, and prevent failures. Logs are also used by security providers like AlienVault and ArcSight to detect security threats and malicious behaviors.

Although logging has long been recognized as an important aspect of software development, it remains predominantly an ad hoc practice. There are no standards, guidelines, or recognized best practices for logging. In addition, software developers continue to insert logs into a system without automated guidance. The lack of systematic and automated approaches for logging has a tremendous impact on the quality of logs, often hindering viable analysis. Log analysis is further complicated by the sheer volume and variability of log data. As a direct consequence, many failure diagnosis tasks are very challenging and expensive to implement, putting at risk the reliability and security of critical systems on which society depends in areas like health, telecom, public safety, and defence.

The ultimate goal of this research program is to advance the practice of logging in software engineering. Much of the existing research effort in the field is devoted to the analysis of logs after they are generated. Considerably less attention is devoted to the study of the practice of logging. Over the next five years, we will focus on four priority research directions. First, we will identify systematic approaches and best practices for logging by studying the current practices in the field. Second, we will develop a context-aware recommendation system to automatically provide logging suggestions to developers. In the third project, we will facilitate the analysis of logs by reducing their complexity. Finally, we will produce techniques to support better logging decisions based on system design and architecture.

The outcomes of this research program are expected to have a real impact on the software industry. Improving logging practices of software developers will result in superior failure diagnosis techniques while reducing the associated costs. Another important impact of this research is its potential for commercialization driven by the strong need from industry to have better logging tools in areas like IoT, cloud computing, and security. Furthermore, the proposed program will train several HQPs who will contribute to the growth of Canada’s ICT sector–an important enabler to Canada’s competitiveness in the world economy.