News
System Tests & Bug Challenge & Config Creator
Written on 29.09.2022 13:05 by Thomas Bock
Dear participants,
From now on, we run our basic tests on your implementation as well as your tests on our reference implementation. Our basic tests test simple behavior of your server. Next week, we will add more advanced tests as well and run your tests against our mutants. Please note that we only test changes that you have pushed to your main branch. Please also note that we stop running the tests as soon as your server crashes 10 times (SERVER_FAIL).
If you think that one of your failing tests is actually correct, you can submit a bug challenge for your team. To do so, please open a ticket in our ticket system and briefly describe your problem. The ticket must contain the name of your test, as well as a description of the test. The ticket system is linked in the CMS under Information. (Tickets that do not belong to the bug challenge will be ignored.) Provided you submit a correct bug challenge, you will get an additional test run on the server as well as the opportunity to submit more challenges on the same day. If it is a buggy test, you will not be able to submit a new challenge that day, but you will get feedback on the bug of your test.
To make testing easier for you, Niklas and Julian have written a Config Creator that allows you to easily create configuration files for your tests. The Config Creator is linked under Information in the CMS. There will be also a brief explanation in the forum on how to use it.
Best,
The SE-Lab team