Big Data Engineering

Dear students,

since we now entered the results of the project, the first students are officially admitted to the exams. You can see your current admission status on your personal status page.

the main exam will take place on 29.07.2025 from 12:00 - 14:00 in the lecture halls E2 2 - HS 0.01 (Günter-Hotz-Hörsaal), E1 3 - HS 002, and E2 5 - HS I. We will announce which lecture hall you have been assigned to via the CMS on your personal status page after the registration deadline in the week before the exam.

The exam will start at 12:00am and will last 120 minutes. Therefore, please be in front of your respective lecture hall at 11:45am at the latest so that we can start on time.

The registration for the exam depends on your course of study. On your personal status page you can see whether you have to register via LSF/HISPOS or the CMS (or FlexNow). For LSF/HISPOS and CMS, you can register as well as withdraw your registration until one week before the exam (22.07.2025). This means, if you plan to participate in the exam, it is best to register right away and if necessary, withdraw your registration again in time. We will import the LSF/HISPOS as well as the FlexNow registrations into the CMS after the registration deadline.

Please note that there are individual registrations for the main exam and the re-exam (with the exception of FlexNow where you automatically register for both exams).

Also note that the registration in CMS is only possible once you have successfully obtained the admission (50% of the mini-test points, 50% of the project points, and no more than 2 mini-tests with 0 points). If there are problems regarding the CMS registration, please contact Luca Gretscher.

In case of problems with the exam registration in general, please contact your responsible examination office.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 09. Don't be confused, you didn't miss Assignment 08, we just wanted to keep the naming consistent with the lecture titles ;-) The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

just for clarification as we received multiple mails on that issue: the code review registration is a team registration, however, as CMS does not support this, you may not see your registration reflected on your personal status page. Therefore, we uploaded the time schedule as mentioned in the last news. All groups are registered correctly, you must attend the code review in your team’s time slot to receive points for the project.

Kind regards,
Your BDE Team

Dear students,

we have just published the timetable for the project code reviews this week under Materials -> Project. Please be there with the whole team at least 10 minutes before your actual timeslot and wait in front of the indicated room until you are asked to enter!

In the code reviews, we will ask you multiple questions regarding the project, mainly about tasks T6-T8 but also about T1-T5 and the bonus tasks T9-T10. Keep in mind that every team member must know and understand all code. We will pick a random team member to answer a specific question. Team members who are not able to explain parts of the code do not get points for the project.

If all team members and the tutor present agree, code reviews may be held in German or in English.

You do not need to bring a laptop with your code, we will show your submission on our own laptops. Please bring your student ID for checking your identity!

Kind regards,
Your BDE Team

Dear students,

we would like to remind you that the deadline for submitting the project and registering for a code review slot is next Sunday, 29.06.25, 23:59. Late submissions and submissions with group sizes other than 3 to 4 students will not be graded!

Also please note that only one team member is allowed to register for the entire team, so please coordinate internally! We just removed all duplicated registrations by hand.

Additionally, here are some clarifications:
  •  The bonus points of tasks T9 and T10 are only for the project, not for the final grade of the entire course.
  •  The code review is mandatory for all team members regardless of the number of points achieved through the unit/end-to-end tests.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded the project under Materials -> Project. A detailed project description including instructions for the submission can be found in the Project Description and the project.pdf file in the root directory of the project (same file). The submission deadline for the project is Sunday, 29.06.25, 23:59! If you correctly joined a team, only one team member has to upload your submission. Please make sure that every team member is able to see the uploaded submission on their personal status page. Only then, all team members will receive the points.

Additionally, each group must participate in a code review after the project submission. We offer seven code review slots:
   •  Slot 1: Wednesday 02.07.25, 10:00 - 12:00
  •   Slot 2: Wednesday 02.07.25, 12:00 - 14:00
  •   Slot 3: Wednesday 02.07.25, 14:00 - 16:00
  •   Slot 4: Wednesday 02.07.25, 16:00 - 18:00
  •   Slot 5: Thursday     03.07.25, 12:00 - 14:00
  •   Slot 6: Thursday     03.07.25, 14:00 - 16:00
  •   Slot 7: Thursday     03.07.25, 16:00 - 18:00

More detailed information about the code reviews will be published at the end of the project, but keep in mind that every team members must know and understand all code. Team members who are not able to explain parts of the code do not get points for the project.

Please register for the code review slot on your personal status page by Sunday, 29.06.25, 23:59! Note that only one team member is allowed to register for the entire team, so please coordinate internally! Registration is on a first-come-first-served basis.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 06. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we would like to remind you that the deadline for creating or joining a team for the project submission is tomorrow at 23:59. This registration is mandatory to participate in the project! 

If you are still looking for a team, you can use the forum. If you are not able to find a team in the forum or somehow else, please still create a team in the CMS for yourself. We will merge teams with less than three people after the deadline and inform you about your team members.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 07. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 05. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

the programming project will start next week. We allow submitting the project in teams of 3 to 4 students. Please note, that submissions of smaller groups will not be graded and thus the exam admission cannot be achieved.

To register your team, you have to use the Team Groupings functionality of the CMS. This means for you that a team member creates a group in the CMS on the personal status page and invites all other team members to the group. You can create your team until Wednesday 11.06.25 at 23:59, after that it is not possible to change the team.

To find a team, you can use our forum.

Kind regards,
Your BDE Team

Dear students,

since the number of participants in the tutorials is continuously decreasing, we have decided to merge tutorials that take place at the same time from now on. Here is an overview of the tutorial slots starting this week.

    •    Tutorial 1 & 2: Wednesday, 10:15 in E1 1, 3.06
    •    Tutorial 3 & 4: Wednesday, 12:15 in E1 1, 3.06
    •    Tutorial 5 & 6: Wednesday, 14:15 in E1 1, 3.06
    •    Tutorial 7 & 8: Wednesday, 16:15 in E1 1, 3.06

You can find a concise overview in our Timetable as well.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 06. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 05. The solutions will be published on Tuesday next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 04. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 04. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 03. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 03. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 02. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 02. The solutions will be published after the mini-test next week.

Note that there is a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 01. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

the mini-test slot registration is now closed. The first mini-test will take place tomorrow, 12:15 - 12:45 in E2 2 - lecture hall 0.01 (Günter-Hotz). More information can be found in the Timetable.

General Information and Procedure

• There are two mini-test slots. The first slot will start at 12:15, the second slot will start once the first one has finished, i.e., approximately at 12:30. Both slots will last 15 minutes.
• You are only allowed to participate in the slot you have registered.
• Be in front of the lecture hall at least 5 minutes early. Please wait until you are instructed to enter the lecture hall. Note that we will ask you to enter the lecture hall through the lower doors and leave through the upper doors to avoid communication between the two slots.
• Use a document-safe, permanent pen such as a ballpoint pen. No additional materials are permitted. If necessary, we provide additional paper.
• As the mini-tests do not have a cover page, we will distribute the mini-tests to the individual seats once you are seated. Please leave every second seat and every third row empty.
• Please fill in your name, your matriculation number, and your tutorial number on the first page. We need your tutorial number to make sure you are able to inspect your mini-test in your tutorial.
• You can start working on the mini-test as soon as you receive a copy and continue until we collect your copy again. We will hand out and collect the mini-tests in the same order so that everyone gets the same amount of processing time. Continued writing after we have asked you to stop to collect your mini-test will be considered as attempted cheating.
• Take your student ID card with you to your seat.

Kind regards,
Your BDE Team

Dear students,

we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start this week. More information can be found in the Timetable.

If you were assigned to Tutorial 9 (Wednesday, 00:00 No Tutorial) but still want to attend a tutorial at some point, you are welcome to attend any other tutorial.

We will upload the tutorial slides before each tutorial and the solutions afterwards.

Kind regards,
Your BDE Team

Dear students,

we would like to remind you to register for a mini-test slot.

Currently, we offer two mini-test slots at the following times:
    •    Slot 1: Tuesday, 12:15 - 12:30
    •    Slot 2: Tuesday, 12:30 - 12:45

Registration is mandatory to attend the mini-tests and works on a first-come-first-served basis. Depending on the number of students, we will add a third slot from 12:45 - 13:00 or in parallel to one of the other two slots, s.t. everyone who wants to attend the mini-tests will be able to register.

More information about the mini-test procedure will follow after the registration has been closed.

Please register for a mini-test slot on your personal status page by Sunday, 20.04.2025, 23:59!

Kind regards,
Your BDE Team

Dear students,

we would like to remind you to set your preferences for the tutorials starting next week.

We offer eight regular tutorials at the following times:
    •    Slot 1: Wednesday, 00:00 No Tutorial: This is a dummy tutorial! Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices.
    •    Slot 2: Wednesday, 10:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E1 1, 2.06
    •    Slot 3: Wednesday, 12:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E2 5, U.36
    •    Slot 4: Wednesday, 14:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E2 5, U.36
    •    Slot 5: Wednesday, 16:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E2 5, U.36

In the tutorials, you have the opportunity to briefly discuss the solution of the previous assignment sheet and to inspect your mini-test. However, the main focus will be a recap of the lecture material by providing additional exercises. Please note, that both the solutions of the assignment sheets as well as the tutorial slides including solutions will be timely uploaded to our Materials.

Because of this, experience has shown that some students will not attend the tutorials in person. Therefore, we also created an additional (pseudo) tutorial (Slot 1: Wednesday, 00:00). Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices. With this, we can allow students that plan to attend, to more likely get their preferred choice. Furthermore, we can better assess how many in-person tutorials we really need. Of course, you can always attend a tutorial if you change your mind.

Please submit your tutorial preferences on your personal status page by Sunday, 20.04.2025, 23:59!

Kind regards,
Your BDE Team

Dear students,

in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine using Docker that has all the required software and systems already installed. You can find detailed instructions at the following Github repository. The link is also available under Information > Docker.

We spent a lot of time preparing the tools and the instructions. Please read them carefully before asking questions.

Please note, that you are also able to run most of the notebooks by just having a Python installation with the required packages. For more information see Information > Manual Installation.

If you still have problems with the installation, you can always ask a question in the forum (BDE25 > Technical). Additionally, there will be a special Docker support office hour on Tuesday, 15.04.2025, 12:15 - 14:00 in E2 2 - Lecture Hall 0.01 (Günter-Hotz).

Kind regards,
Your BDE Team