News
Important: Pipeline Updates
Written on 02.10.2023 16:55 by Kallistos Weis
Dear Participants,
In order to provide more feedback about your implementation to you, we have decided to split the pipeline into three parts.
The first part will run our system tests on your implementation and part of your system tests on our implementation. You need to register these system tests in the SystemTestRegistration class in a function called 'registerSystemTestsReferenceImpl' in the SystemTestRegistration.kt file in the package systemtest. You can use this category for system tests for which you want to check the simulation behaviour of our reference implementation and to fix your system tests before including them in different parts of the pipeline.
The second part will run your system tests for the validation process on our reference implementation and, if all tests pass, run these system tests on mutants of our reference implementation in which we have introduced bugs regarding the validation of the configuration files. This second part is only concerned with the validation process of the configuration files, and will therefore only result in possibly incorrect
log messages before the 'Simulation start' message (i.e. the simulation behaves exactly as in the reference implementation). You need to register them in the SystemTestRegistration class in a function called 'registerSystemTestsMutantValidation' in the SystemTestRegistration.kt file in the package systemtest.
The third part will run your system tests for the simulation on our reference implementation and, if all tests pass, run these system tests on mutants of our reference implementation in which we have introduced bugs for the simulation of the scenario. The third part is only concerned with the simulation process itself, and thus will only result in the wrong log messages after the 'Simulation start' message (i.e. the validation process behaves exactly as in the reference implementation). You need to register them in the SystemTestRegistration class in a function called 'registerSystemTestsMutantSimulation' in the SystemTestRegistration.kt file in the package systemtest.
As the last two parts are independent of each other, you should only register your system tests for one part, as validation tests will not find any bugs in the simulation and vice versa.
The results of each part will be sent to you separately via email.
Note: Make sure you update the registration of your system tests correctly, otherwise you will not receive any results for your system tests.
By splitting the test pipeline into three separate parts, we also increase the number of runs, so you receive test results more frequently.
You will be able to trigger the pipeline for the first part by pushing to the 'main' branch of your repository about every 2 hours.
You will be able to trigger the pipeline for the second part by pushing to the 'main' branch of your repository about every 8 hours.
You will be able to trigger the pipeline for the third part by pushing to the 'main' branch of your repository about every 8 hours.
Best,
The SE-Lab Team