News
Test Pipeline Release
Written on 18.09.2024 16:43 by Kallistos Weis
Dear participants,
From now on, we run the set of tests that are mandatory to pass on your implementation as well as your tests on our reference implementation (using the function called 'registerSystemTestsReferenceImpl' in the SystemTestRegistration.kt file).
We have three categories of tests: validation tests, component tests, and full tests.
The validation tests check how you handle correct and incorrect configuration files,
component tests check the behavior of your implementation for specific parts of the simulation, and
full tests check the behavior of your implementation for the whole simulation.
Please note that we only test changes that you have pushed to your main branch.
Please also note that we stop running the tests as soon as your server crashes 10 times (SERVER_FAIL or TIME_OUTS).
On Monday, we will release a second set of tests to the three categories that will be optional for passing the course but help to debug your simulation.
Additionally starting on Monday, we run your tests (registered in the function 'registerSystemTestsMutantSimulation' in the SystemTestRegistration.kt file) against mutants of the reference implementation to check whether your tests are able to detect the introduced faults.
However, we will only run your tests on our mutants, if all tests registered in this function succeed on our reference implementation.
Important: You have to pass at least 97 out of the 100 mandatory tests to be able to pass the group phase.
You can find a document with all the mandatory tests in the Materials (CMS) under General with a short description of the test scenarios.
Best,
Your SE-Lab Team