Until the 2000s, building any software had been a two-step process: developing the code (software development), followed by deployment and configuration (IT operations). These processes used to be managed by separate teams, with different priorities and KPIs.
The past decade has seen these two roles merge into DevOps, an agile development methodology characterised by integrated teams working on both software development and IT operations at the same time. From an operational standpoint, DevOps is based on merging different parts of code early in the development process, and using automated testing, lately with the support of artificial intelligence (AI), to validate every change in the code as it gets corrected, adapted, and deployed.
However, modern software ecosystems are becoming increasingly complex, often integrating cyber-physical systems (CPS), such as those that drive mechanisms, vehicles, and production lines, to name a few. Applying DevOps to these systems presents a new host of challenges that software practitioners have to face, all with far-reaching consequences. This is something the EU has decided to investigate through the DevOps for Complex Cyber-physical Systems project under the Horizon 2020 programme. The project is driven by a consortium of entities including the University of Luxembourg’s Interdisciplinary Centre for Security, Reliability and Trust (SnT).
“The innovations developed by the University of Luxembourg are being assessed on three different use-case scenarios, representative of major CPS domains (e.g., avionics, automotive, and railway systems),” says Prof. Domenico Bianculli, chief scientist II/associate professor in software engineering at SnT’s Software Verification and Validation (SVV) research group, who coordinates the project on the University’s side. “For example, the avionics use case deals with the development of free route airspace algorithms, such as finding an optimal route with the shortest distance between two waypoints and the lowest fuel consumption,” he explains.
In the framework of the project, the SnT team has developed both a novel testing technique that supports different objectives (e.g. assess robustness to adversarial actors, or testing how estimated fuel consumption may vary depending, say, on wind speed), as well as a runtime verification technique that will ensure that the timing-based behaviour of the route finder is as expected, and that its resource consumption is reasonable. A task far from being simple, given the stakes at play, the wide range of scenarios to take into account, and the time-cost constraints of DevOps.
“We were met with three main challenges: first, adaptability, i.e., defining a solution that works with different simulator types and with hardware-in-the-loop,” says Prof. Bianculli. “Secondly, cost-effectiveness, since our verification and validation solution should fit within the intrinsic time limits of the DevOps processes. Thirdly, guaranteeing efficient and safe testing when physical devices are involved, since a limited number of inputs can be tested, and illegal inputs may damage hardware or cause safety hazards,” he explains.
In addition, working with CPS increases the number of scenarios to test, as a large number of real-life factors (e.g., weather) can affect their performance compared to testing an ICT system. “For example, an autopilot for an unmanned aerial vehicle may cause a crash if strong top wind follows a mild front wind after a rainstorm,” says Prof. Fabrizio Pastore, chief scientist II/associate professor in software engineering at SnT and member of the project team. “With CPSs, we may easily end up with a combinatorial explosion of the number of scenarios to test, and could overlook some conditions. To efficiently identify this kind of scenario, we employ a reinforcement learning algorithm. By observing how certain events (e.g., increase in wind speed) affect the functioning of the system under test (e.g., the vertical acceleration of the autopilot decreases) the algorithm learns which events are more successful in decreasing the performance of the system under test, and combines them until a safety problem is identified,” he explains.
To address the security analysis of CPSs, the SnT team has also developed a specification language, E-iCFTL, that helps find vulnerabilities in software. E-iCFTL allows developers to describe executions of the system under test when there are no vulnerabilities. Then, the framework developed at SnT inserts instrumentation code into the system under test, which generates a trace for each execution. These traces are then analysed by the framework over multiple executions. Any differences between the traces and the specifications defined by the developers indicates a security vulnerability, which is then further inspected and processed by the framework, that then estimates the possible causes of the vulnerability.
The project, which started in January 2021, has just entered its final year. The consortium is expected to deliver their final results by December 2023.