News
Tutorial AssignmentWritten on 26.04.24 by Luca Gretscher Dear students, we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start next week. More information can be found in the Timetable. If you were assigned to Tutorial 9 (Monday, 00:00 No Tutorial) but still want to… Read more Dear students, we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start next week. More information can be found in the Timetable. If you were assigned to Tutorial 9 (Monday, 00:00 No Tutorial) but still want to attend a tutorial at some point, you are welcome to attend the tutorial on Tuesdays 12:15 - 13:45 in the Günter-Hotz lecture hall. Please bring your laptop to the tutorials since we will do some coding exercises (at least next week). Additionally, we will upload the tutorial slides before each tutorial. Kind regards, |
Assignment 01 & Team GroupingsWritten on 25.04.24 by Luca Gretscher Dear students, we have just uploaded Assignment 01. Instructions for the submission can be found at the end of the document. Please note that the submission is only allowed in teams of 3 to 4 students. Furthermore, we ask that you try to form teams of exactly 4 students to reduce the number of… Read more Dear students, we have just uploaded Assignment 01. Instructions for the submission can be found at the end of the document. Please note that the submission is only allowed in teams of 3 to 4 students. Furthermore, we ask that you try to form teams of exactly 4 students to reduce the number of submissions. To register your team, you have to use the If you haven't found a team yet, you can use our forum to find one. Kind regards, |
Reminder: Tutorial PreferencesWritten on 25.04.24 by Luca Gretscher Dear students, We offer nine regular tutorials at the following times: Dear students, We offer nine regular tutorials at the following times: In the tutorials, we will in general briefly discuss the solution of the previous assignment sheet. However, the main focus will be a recap of the lecture material by providing additional exercises. Because of this, experience has shown that some students will not attend the tutorials in person. Therefore, we also created an additional (pseudo) tutorial (Slot 1: Monday, 00:00). Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices. With this, we can allow students that plan to attend, to more likely get their preferred choice. Furthermore, we can better assess how many in-person tutorials we really need. Of course, you can always attend a tutorial if you change your mind. Please submit your tutorial preferences on your personal status page by 23:59 today! Kind regards, |
Vagrant & DockerWritten on 24.04.24 by Luca Gretscher Dear students, in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine using Vagrant that has all the required software and… Read more Dear students, in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine using Vagrant that has all the required software and systems already installed. You can find detailed instructions at the following Github repository. The link is also available under Information > Vagrant. We also provide a small FAQ. As there are problems with Vagrant on Apple Silicon chips, we also created a Docker image which can be used instead of the Vagrant VM. You can find detailed instructions at the following Github repository. The link is also available under Information > Docker. We spent a lot of time preparing the tools and the instructions. Please read them carefully before asking questions. Please note, that you are also able to run most of the notebooks by just having a Python installation with the required packages. If you still have problems with the installation, you can always ask a question in the forum (bde24 > Technical Problems). Kind regards, |
Big Data Engineering
This course covers the fundamentals of data management and query processing.
- Learn basic techniques in "Big Data Engineering" conceptually.
- Learn to apply basic techniques in "Big Data Engineering".
- Help you not to reinvent the wheel later.
- Learn to map new problems to existing problems and solve them using established techniques.
- Raise awareness for problems of important applications: Privacy, deanonymization, ethical issues.
- Raise awareness for solutions of important applications: Effort, performance, robustness, extensibility, maintainability.
Lecture
The lecture takes place every Thursday at 10:15 in building E2 2 - lecture hall 0.01 (Günter-Hotz-Hörsaal). The recording of the lecture will be uploaded to Youtube afterwards. The lecture was awarded with two busy beaver teaching awards (2011, 2021) and two busy beaver honorable mentions (2022, 2023).
Tutorials
On Mondays and Tuesdays there will be tutorials. In the tutorials there will be both exercises and the possibility to ask questions regarding the assignment sheets and the lecture. Please indicate your preferred tutorial timeslots on your personal status page by 23:59 on 25.04.2024. We will then assign you to your tutorial. If you are unable to attend on your assigned date at some point, you are welcome to attend another tutorial. The tutorials will start in the week of 29.04.2024.
Office Hours
In addition to the tutorials, there will be an Office Hour on Wednesdays from 14:15 to 15:45 in the seminar room E1.1 R3.06. In the Office Hour you can ask questions about the lectures and assignment sheets, as well as get help with technical problems.
Assignment Sheets
During the day after each lecture we publish an assignment sheet with tasks related to the lecture under Materials. These usually consist of three written exercises and one programming exercise. Assignment sheets are to be completed by you in groups of 3-4 students and submitted via the CMS before the start of the next lecture. Afterwards, your submissions will be corrected by our team and you will receive points and feedback via the CMS. More information about the exact dates for the release and submission of the assignment sheets can be found in the Timetable.
Project
Assignment sheets 7 and 8 will be replaced by a small project for which you have to develop a toy backend server for a social network. The project is to be completed by you in groups of 3-4 students (not necessarily the same group as for the assignments). Your points will be determined by functional tests (50%) and the quality of your code together with the outcome of a short code review (50%). Note that all team members must know and understand all code. Team members who are not able to explain parts of the code do not get points for the project.
Exam Admission
In order to be allowed to take part in the written exam and/or the re-exam, at least 50% of the possible points from both the assignment sheets and the project must be achieved each. In addition, a maximum of 2 assignment sheets may be graded with 0 points.
Exam
At the end of the semester there will be both a written exam and a re-exam.
- Exam: 01.08.24, 10:00 - 12:00
- Re-exam: 26.09.24, 14:00 - 16:00
Grade
The better result of the written exam and the re-exam determines 100% of the final grade. In order to pass the lecture, at least 50% of the points must be achieved in the written exam or the re-exam.