News
Test Pipeline Release
Written on 13.10.2023 16:09 by Kallistos Weis
Dear participants,
From now on, we run a first set of tests on your implementation as well as your tests on our reference implementation (using the function called 'registerSystemTestsReferenceImpl' in the SystemTestRegistration.kt file).
We have three categories of tests: validation tests, component tests, and full tests.
The validation tests check how you handle correct and incorrect configuration files,
component tests check the behavior of your implementation for specific parts of the simulation, and
full tests check the behavior of your implementation for the whole simulation.
Please note that we only test changes that you have pushed to your main branch.
Please also note that we stop running the tests as soon as your server crashes 10 times (SERVER_FAIL or TIME_OUTS).
On Monday, we will release a second set of tests to the three categories.
There will be no release of another set of tests after Monday.
Additionally starting on Monday, we run your tests (registered in the function 'registerSystemTestsMutantSimulation' in the SystemTestRegistration.kt file) against mutants of the reference implementation to check whether your tests are able to detect the introduced faults.
However, we will only run your tests on our mutants, if all tests registered in this function succeed on our reference implementation.
Best,
Your SE-Lab Team