News
Scores and Feedback Available for Assignment 1Written on 26.05.25 by Norman Peitek Dear students,
We have completed our individual review of Assignment 1. The feedback and scores are available on the CMS. We will discuss high-level topics, such as common mistakes, in the next Q&A on Wednesday.
Your individual feedback contains two different annotation methods. The… Read more Dear students,
We have completed our individual review of Assignment 1. The feedback and scores are available on the CMS. We will discuss high-level topics, such as common mistakes, in the next Q&A on Wednesday.
Your individual feedback contains two different annotation methods. The red highlighting indicates our point scoring, where we generally use three different annotations: - A checkmark ✔️ indicates correct or full point(s) - A crossed checkmark (similar to an “x”) indicates partially correct, or roughly 0.5/0.25 points depending on the task - An "f" indicates something incorrect Please keep in mind that this is not an exam where the goal is to get full points. The point of the assignments is that you practice important elements of empirical research. You only need to fulfill 50% of the points across all assignments to qualify for the exam, so it is fine if you have missed some points.
The blue highlighting indicates individual feedback for your improvement. Please carefully review those as they will be relevant again for the next two assignments (and the exam).
Some solutions had major issues (e.g., discussing the wrong paper) such that we could not provide meaningful feedback. In those instances, we have left a note that you have the option to re-submit this task via email to get feedback. Please note that this does *not* change your grading in any way, it is just a voluntary option for you to get meaningful feedback.
If you have questions regarding your individual feedback that you would like to discuss one-on-one, Annabelle is offering an office hour on Wednesday, 28th of May, between 11:00 and 12:00 at her office E1.1, Room 2.14. In case you have something you would like to discuss, please write Annabelle a brief email so we can prepare the discussion and minimize wait times.
The next assignment (data analysis) starts on Wednesday. We will briefly introduce the assignment in our live Q&A. You then have three weeks until Tuesday, 17th of June, to work on it.
Best, EMSE-Team |
Q&A Tomorrow Cancelled - Everything Else Stays on ScheduleWritten on 13.05.25 by Norman Peitek Dear students,
Unfortunately, I have to cancel tomorrow's Q&A due to illness. The other parts of the schedule will be unaffected and continue as usual. Thus, the mini-test about experiment design will still happen tomorrow starting at 9:45. The next lecture on data & measures will still be… Read more Dear students,
Unfortunately, I have to cancel tomorrow's Q&A due to illness. The other parts of the schedule will be unaffected and continue as usual. Thus, the mini-test about experiment design will still happen tomorrow starting at 9:45. The next lecture on data & measures will still be available starting from 14:00. There were no submitted questions on the assignment so far. If you had planned to ask something tomorrow during the Q&A, please send either me or Annabelle an email as soon as possible. Just as a reminder, the assignment is due in one week (i.e., next Tuesday, 20th of May). We will make up for the cancelled Q&A on the content of the experiment design lecture next week, so please still feel free to submit questions on the lecture content.
Thanks for understanding and apologies for the inconvenience, Norman |
Welcome to the Course: First Mini-Test and Q&A Tomorrow!Written on 15.04.25 by Norman Peitek Dear students,
We would like to welcome you to Empirical Software Engineering Research. We are excited that you joined this course and hope you already had a chance to watch the kickoff and introduction lectures.
This is a friendly reminder that we have our first mini-test tomorrow… Read more Dear students,
We would like to welcome you to Empirical Software Engineering Research. We are excited that you joined this course and hope you already had a chance to watch the kickoff and introduction lectures.
This is a friendly reminder that we have our first mini-test tomorrow (Wednesday) between 9:45 and 10:15 am on the CMS. Please note that participation in all mini-tests is mandatory for students aiming to earn credits for the course. But, even if you are only auditing the course, feel free to participate in the mini-test as feedback for your own and the groups’ understanding of the concepts. The CMS automatically saves your answers, and there is no explicit “save” or "submit" button. You should see an indicator confirming your responses have been saved.
Later in the day, we are meeting for our first live Q&A, which will start at 12:15 sharp. You can find the Microsoft teams link on the CMS.
To avoid cluttering inboxes, this will be the only time we remind everyone for the upcoming mini-test and Q&A. We recommend downloading the course schedule from the CMS to your calendar and setting up your own reminders depending on your needs.
See you tomorrow, EMSE-Team
|
Empirical Software Engineering Research
Empirical software engineering research plays an important role to improve software engineering by studying its processes, tools, and humans with empirical methods. In this course, we provide an overview of typical quantitative and qualitative research methods. We discuss in-depth on how to properly conduct empirical studies and interpret collected data. Specific topics will include:
- Importance of empirical methods in SE
- Controlled experiments
- Experiment designs, variables, operationalization, and threats to validity
- Data and measures
- Data analysis
- Experiment conduct
- Qualitative studies (e.g., surveys, case studies, interviews)
- Secondary studies (e.g., systematic literature reviews)
- Report writing
- Replication
- Open science
- Ethics
- Understand empirical methods in software engineering (research)
- Awareness of various experiment designs including their (dis)advantages
- Understand different types of data and how to describe and interpret them with statistical methods
- Awareness of critical topics and limitations surrounding empirical methods
Important Note on Online Lecture Format
This course will be entirely online. The lectures will be uploaded as videos. For each lecture, we will offer a live, online Q&A in the subsequent week (Wednesday, 12:15–13:45).
In addition, we have weekly online mini-tests (Wednesday, 9:45–10:15) and three larger assignments, that are a prerequisite to be admitted to the exam.
Details are provided in the kickoff session, which is already available online.
Limited Registration: First-Come-First-Serve and Waiting List
We are offering this course for the first time in this format. Additionally, we have limited staff to advise your learning and provide feedback. Thus, we have to limit the number of students that you can fully participate and take the exam in this semester to a maximum of 30 students.
Nevertheless, we believe that the course material is valuable for more students. Thus, the course registration is open and unlimited so that any student can still access and audit the materials (but unfortunately not obtain credit points).
To make it clear who is fully enrolled in the course, can submit assignments, and also take the exam (if the requirements are met), we display this status as a testing result. This means if you would like to participate in the assignments, take the exam, and get credits for this course, you must be one of the 30 students with the "passed" status in this testing.
Initially, we will enroll the first 30 students registered for the course. All other students will be on our waiting list with the possibility to be later admitted until the start of the first assignment (7th of May 2025).
Therefore, there are these possible scenarios:
- You enrolled very early and secured a spot in our course.
If you want to take the exam, take part in every mini-test and submit the assignments.
If you only want to access the material, do not take part in the mini-tests, and you will be excluded from the exam, opening your spot to the next person on the waiting list. You are still invited to join our Q&A sessions and watch the videos. - You enrolled and only secured a spot on our waiting list. We will upload your position on the waiting list to your personal page, so you can estimate your chances of joining the course. We strongly recommend still taking the mini-tests even before you receive a spot, since you will need them to qualify for the exam.
- In general: If you did not take part in any of the mini-tests, you can not attend the exam and therefore your spot will be opened to the next person on the waiting list.
(In case of illness, please contact as soon as possible via e-mail with a medical certificate.)
To obtain access to the course material, you must register via this CMS course. Registration starts on 1st of April 2025 at 8:00 am and ends on 16th of April 2025 at 11:00 am.
Update: The initial 30 spots have been taken. Everyone joining from now on will be on the waiting list. The displayed waiting list spot on the CMS does not update automatically. We will update it occasionally.
Mini-Tests
There will be 12 mini-tests, each mini-test is worth 4 points. The mini-test will take part on Wednesday 9:45 - 10:15 online, you can take the mini-test within these 30 minutes in the CMS. The mini-test itself has a time limit of 10 minutes, since started. The questions will focus on the lecture of the previous week.
Assignments
There will be three assignments in this course, each worth 20 points. For each, you have two to three weeks to work on. The three assignment topics are:
- Understanding empirical studies
- Data analysis
- Report writing
For each submitted assignment, you will receive personalized feedback to help you improve.
To qualify for the exam, you need to score, at least,
- 50% (24/48 points) on the mini-tests
- 50% (30/60 points) on the assignments
- 100% of the mini-tests must be submitted
- 100% of the assignments must be submitted.
Final Exam
The written exam will be 28.07.2025 HS002 in E1 3 at 10 am. There is written a re-exam on 01.09.2025 - HS003 in E1 3 at 12 am. Each exam will last 60 minutes.
You will need to register in the LSF/Hispos for the exam until 21.07.2025 for the exam and 25.08.2025 for the re-exam. We cannot help if you miss the registration deadline.
Prerequisites
There are no formal prerequisites. An understanding of the basics of software engineering is beneficial.
Literature
The library is providing access to the following books:
- Wohlin et al., Experimentation in Software Engineering, Springer, 2012/2024
- Juristo and Moreno, Basics of Software Engineering Experimentation, Springer, 2001
- Felderer and Travassos, Contemporary Empirical Methods in Software Engineering. Springer, 2020.
- Prechelt, Kontrollierte Experimente in der Softwaretechnik: Potenzial und Methodik, Springer, 2001/2014