Big Data Engineering Prof. Dr. Jens Dittrich Basic Lecture (6 CP), Summer Semester 2023

News

02.06.2023

Merging of Tutorials

Dear students,

since the number of participants in the tutorials is continuously decreasing, we have decided to merge some of the tutorials that take place at the same time. Concretely, we will be merging the tutorials 1 & 2 and the tutorials 3 & 4 from now on.... Read more

Dear students,

since the number of participants in the tutorials is continuously decreasing, we have decided to merge some of the tutorials that take place at the same time. Concretely, we will be merging the tutorials 1 & 2 and the tutorials 3 & 4 from now on. Here is an overview of the tutorial slots starting next week.

    •    Tutorial 1 & 2: Monday, 10:15 in E1.1 R3.06
    •    Tutorial 3 & 4: Monday, 12:15 in E1.1 R3.06
    •    Tutorial 5: Tuesday, 12:15 in E1.1 R3.06
    •    Tutorial 6: Tuesday, 12:15 in E1.1 R2.06
    •    Tutorial 7: Tuesday, 10:15 Online on our Discord server

You can find a concise overview in our Timetable as well.

Kind regards,
Your BDE Team

01.06.2023

Cancellation Lecture on 08.06.2023

Dear students,

due to the public holiday (Fronleichnam) next Thursday, 08.06.2023, the lecture will be cancelled on that day.

The next lecture will be held on Thursday, 15.06.2023.

Kind regards,
Your BDE Team

01.06.2023

Assignment Sheet 06

Dear students,

we have just uploaded Assignment 06. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

25.05.2023

Assignment Sheet 05

Dear students,

we have just uploaded Assignment 05. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

25.05.2023

Cancellation Tutorials on 29. & 30.05.2023

Dear students,

due to the public holiday (Pfingstmontag) next Monday, 29.05.2023, and because we do not have an assignment sheet to discuss, all tutorials (Monday and Tuesday) next week will be cancelled.

The office hour on Wednesday will take place as... Read more

Dear students,

due to the public holiday (Pfingstmontag) next Monday, 29.05.2023, and because we do not have an assignment sheet to discuss, all tutorials (Monday and Tuesday) next week will be cancelled.

The office hour on Wednesday will take place as usual.

Kind regards,
Your BDE Team

12.05.2023

Cancellation Lecture on 18.05.2023

Dear students,

due to the public holiday (Christi Himmelfahrt) next Thursday, 18.05.2023, the lecture will be cancelled on that day.

The next lecture will be held on Thursday, 25.05.2023.

Kind regards,
Your BDE Team

11.05.2023

Assignment Sheet 04

Dear students,

we have just uploaded Assignment 04. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

10.05.2023

Reminder: Lecture Hall 11.05.2023

Dear students,

due to the Eurographics conference this week, our lecture hall is not available tomorrow.
Threfore, the lecture will take place in building A5.1, lecture hall I (-1.03).

Here is a map with an overview of all buildings of the university.

Kind... Read more

Dear students,

due to the Eurographics conference this week, our lecture hall is not available tomorrow.
Threfore, the lecture will take place in building A5.1, lecture hall I (-1.03).

Here is a map with an overview of all buildings of the university.

Kind regards,
Your BDE Team

05.05.2023

Lecture Hall & Seminar Rooms Next Week (08.05. - 14.05.)

Dear students,

due to the Eurographics conference next week, our lecture hall and seminar rooms are not available.
Therefore, we will switch to the following rooms.
 

Lecture Hall

The lecture next Thurday, 11.05.2023, will take place in building A5.1,... Read more

Dear students,

due to the Eurographics conference next week, our lecture hall and seminar rooms are not available.
Therefore, we will switch to the following rooms.
 

Lecture Hall

The lecture next Thurday, 11.05.2023, will take place in building A5.1, lecture hall I (-1.03).
Here is a map with an overview of all buildings of the university.
 

Tutorials

The tutorials will take place in the following rooms.

  • Tutorial 1: Monday, 10:15 - 12:00 in E1.3, R 0.14
  • Tutorial 2: Monday, 10:15 - 12:00 in E1.1, R 1.06
  • Tutorial 3: Monday, 12:15 - 14:00 in E2.5, R 4 (U.16) (Math)
  • Tutorial 4: Monday, 12:15 - 14:00 in E1.1, R 1.06
  • Tutorial 5: Tuesday, 12:15 - 14:00 in E2.5, R 4 (U.16) (Math)
  • Tutorial 6: Tuesday, 12:15 - 14:00 in E2.1, R 0.01 (Bioinformatics)
  • Tutorial 7: Tuesday, 10:15 - 12:00 as usual online on our Discord server


Office Hour

Since there is no seminar room available at our usual time slot, the Office Hour will be held online on our BDA Discord server.

You can find an overview in the Timetable as well.

Kind regards,
Your BDE Team

04.05.2023

Assignment Sheet 03

Dear students,

we have just uploaded Assignment 03. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

27.04.2023

Postponement Tutorials 01.05.23

Dear students,

due to the public holiday next Monday, 01.05.2023, the tutorials will be cancelled on that day.

We will therefore offer alternative tutorial slots:
    •    Tutorial 1: Wednesday, 03.05.23 at 16:15 - 18:00 in E1.1, R 3.06
    •    Tutorial 2:... Read more

Dear students,

due to the public holiday next Monday, 01.05.2023, the tutorials will be cancelled on that day.

We will therefore offer alternative tutorial slots:
    •    Tutorial 1: Wednesday, 03.05.23 at 16:15 - 18:00 in E1.1, R 3.06
    •    Tutorial 2: Wednesday, 03.05.23 at 10:15 - 12:00 in E1.1, R 3.06
    •    Tutorial 3: Wednesday, 03.05.23 at 16:15 - 18:00 in E1.1, R 2.06
    •    Tutorial 4: Wednesday, 03.05.23 at 12:15 - 14:00 in E1.1, R 3.06

You can find a concise overview in the Timetable as well.

Please try to attend your assigned tutorial. If this is not possible for you, you may attend another tutorial as long as there are available seats on-site.

Kind regards,
Your BDE Team

27.04.2023

Assignment Sheet 02

Dear students,

we have just uploaded Assignment 02. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

26.04.2023

Reminder: Team Groupings

Dear students,

we would like to remind you that the deadline for creating or joining a team for the assignment submissions is tomorrow morning at 10am. If you are still looking for a team, you can use the forum. There are still open threads.

Please note, that... Read more

Dear students,

we would like to remind you that the deadline for creating or joining a team for the assignment submissions is tomorrow morning at 10am. If you are still looking for a team, you can use the forum. There are still open threads.

Please note, that only submissions in groups of 2 to 3 students are allowed. Individual submissions will not be graded.

Kind regards,
Your BDE Team

24.04.2023

Manual Installation of Python Packages

Dear students,

as already announced, we will make use of Jupyter notebooks throughout this lecture. To run these notebooks, we require a Python installation and a couple of Python packages. In addition, a few of the notebooks also require third-party software... Read more

Dear students,

as already announced, we will make use of Jupyter notebooks throughout this lecture. To run these notebooks, we require a Python installation and a couple of Python packages. In addition, a few of the notebooks also require third-party software such as PostgreSQL or Neo4j. Therefore, we provide you with a virtual machine that already has everything installed.
Unfortunately, some of you encounter problems regarding the installation and/or use of the virtual machine, e.g. the installation does not work on MacBooks with M1 chips.

However, you do not necessarily need the virtual machine! For the assignment sheets, you will only need Python and the corresponding Python packages (no third-party software). Thus, if you are unable to use the virtual machine for whatever reason, you can also manually install the required software.
You can find instructions on how to manually install Python and the Python packages here. The link is also available under Information > Manual Installation.

Kind regards,
Your BDE Team

24.04.2023

Reminder: Tutorial Room Change

Dear students,

please note, that the seminar room 2.06 in building E1.1 is otherwise occupied today. Therefore, Tutorial 2 (10:15 - 12:00) and Tutorial 4 (12:15 - 14:00) take place in building E1.7 (Cluster of Excellence) R 001.

Kind regards,
Your BDE Team

21.04.2023

Tutorial Assignment

Dear students,

we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start next week. More information can be found in the Timetable.

If you were assigned to Tutorial 8 (Sunday,... Read more

Dear students,

we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start next week. More information can be found in the Timetable.

If you were assigned to Tutorial 8 (Sunday, 00:00 NO Tutorial) but still want to attend a tutorial at some point, you are welcome to attend the online tutorial on Tuesdays 10:15 - 12:00 on our Discord server.

Notice!
Tutorial 2 (10:15-12:00,  E1.1 R 2.06) and Tutorial 4 (12:15-14:00, E1.1 R 2.06) next week Monday 24.04.2023 will be held in building E1.7 R 001.
The usual seminar room 2.06 in building E1.1 is occupied on Monday.

Kind regards,
Your BDE Team

20.04.2023

Assignment Sheet 01 & Team Groupings

Dear students,

we have just uploaded Assignment 01. Instructions for the submission can be found at the end of the document.

Please note that the submission is only allowed in teams of 2 to 3 students. Furthermore, we ask that you try to form teams of exactly... Read more

Dear students,

we have just uploaded Assignment 01. Instructions for the submission can be found at the end of the document.

Please note that the submission is only allowed in teams of 2 to 3 students. Furthermore, we ask that you try to form teams of exactly 3 students to reduce the number of submissions.

To register your team, you have to use the Team Groupings functionality of the CMS. This means for you that a team member creates a group in the CMS on the personal status page and invites all other team members to the group. You can create your team until Thursday 27.04.23 at 10:00am, after that it is not possible to change the team. If you need to change your team during the semester, please contact your tutor.

If you haven't found a team yet, you can use our forum to find one.

Kind regards,
Your BDE Team

20.04.2023

Reminder: Tutorial Preferences

Dear students,

we would like to remind you that you can change your preferences for the tutorials only until tonight.
Please note that we had to reset your initial preferences on Monday, 17.04.23, due to the addition of the online tutorial.

You can find a... Read more

Dear students,

we would like to remind you that you can change your preferences for the tutorials only until tonight.
Please note that we had to reset your initial preferences on Monday, 17.04.23, due to the addition of the online tutorial.

You can find a detailed overview of the available time slots in this previous news post.

Please submit your tutorial preferences on your personal status page by 23:59 today!

Kind regards,
Your BDE Team

17.04.2023

Tutorials

Dear students,

due to significantly more registrations for our lecture this year, we decided to offer one additional online tutorial. This tutorial will be held on our Discord server every Tuesday at 10:15 - 12:00. You can find more information about our Discord... Read more

Dear students,

due to significantly more registrations for our lecture this year, we decided to offer one additional online tutorial. This tutorial will be held on our Discord server every Tuesday at 10:15 - 12:00. You can find more information about our Discord server under Information > Discord.

We now have the following tutorial slots:
    •    Slot 1: Sunday, 00:00 NO Tutorial: This is a dummy tutorial! Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices.
    •    Slot 2: Monday, 10:15 On-site Tutorial: Two regular tutorials at this time slot in E1.1 R2.06 and E1.1 R3.06
    •    Slot 3: Monday, 12:15 On-site Tutorial: Two regular tutorials at this time slot in E1.1 R2.06 and E1.1 R3.06
    •    Slot 4: Tuesday, 10:15 Online Tutorial: Online tutorial on our Discord server.
    •    Slot 5: Tuesday, 12:15 On-site Tutorial: Two regular tutorials at this time slot in E1.1 R2.06 and E1.1 R3.06

Note, that by adding the online tutorial, we had to reset your initial preferences!

Therefore, we extended the deadline for the tutorial preferences by one day (20.04.2023, 23:59).


Kind regards,
Your BDE Team

13.04.2023

Vagrant & Virtual Machine

Dear students,

in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine that has all the... Read more

Dear students,

in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine that has all the required software and systems already installed. You can find detailed instructions at the following Github repository. The link is also available under Information > Vagrant.

We spent a lot of time preparing the tools and the instructions. Please read them carefully before asking questions. You can also consult the threads under bde22 > Technisches in our forum and check previously asked questions. We also provide a small FAQ.

Please note, that you are also able to run most of the notebooks by just having a Python installation with the required packages.

If you still have problems with the installation, you can either ask a question in the forum (bde23 > Technical Problems) or visit our Office Hour on Wednesday, 19 April at 14:15.

Kind regards,
Your BDE Team

12.04.2023

Tutorial Preferences

Dear students,

you can now choose your preferred tutorial slots on your personal status page. You can change your preferences until 19.04.23 at 23:59.

We offer six regular tutorials at the following times.
    •    2x Monday, 10:15 - 12:00
    •    2x... Read more

Dear students,

you can now choose your preferred tutorial slots on your personal status page. You can change your preferences until 19.04.23 at 23:59.

We offer six regular tutorials at the following times.
    •    2x Monday, 10:15 - 12:00
    •    2x Monday, 12:15 - 14:00
    •    2x Tuesday, 12:15 - 14:00

In the tutorials, we will in general briefly discuss the solution of the previous assignment sheet. However, the main focus will be a recap of the lecture material by providing additional exercises.
Please note, that both the solutions of the assignment sheets as well as the tutorial slides including solutions will be timely uploaded to our Materials.

Because of this, experience has shown that some students will not attend the tutorials in person. Therefore, we also created an additional (pseudo) tutorial (Slot 1: Sunday, 00:00). Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices. With this, we can allow students that plan to attend, to more likely get their preferred choice. Furthermore, we can better assess how many in-person tutorials we really need. Of course, you can always attend a tutorial if you change your mind.

Kind regards,
Your BDE Team

Show all
 

Big Data Engineering

This course covers the fundamentals of data management and query processing.

Learning objectives of this course are:
  • Learn basic techniques in "Big Data Engineering" conceptually.
  • Learn to apply basic techniques in "Big Data Engineering".
  • Help you not to reinvent the wheel later.
  • Learn to map new problems to existing problems and solve them using established techniques.
  • Raise awareness for problems of important applications: Privacy, deanonymization, ethical issues.
  • Raise awareness for solutions of important applications: Effort, performance, robustness, extensibility, maintainability.

Lecture

The lecture takes place every Thursday at 10:15 am in building E2 2 - lecture hall 0.01 (Günter-Hotz-Hörsaal). The recording of the lecture will be uploaded to Youtube afterwards.

Tutorials

On Mondays and Tuesdays there will be tutorials. In the tutorials there will be both exercises and the possibility to ask questions regarding the assignment sheets and the lecture. Please indicate your preferred tutorial timeslots on your personal status page by 23:59 on 19.04.2023. We will then assign you to your tutorial. If you are unable to attend on your assigned date at some point, you are welcome to attend another tutorial. The tutorials will start in the week of 25.04.2023.

Office Hours

In addition to the tutorials, there will be an Office Hour on Wednesdays from 14:15 to 16:00 in the seminar room E1.1 R3.06. In the Office Hour you can ask questions about the lectures and assignment sheets, as well as get help with technical problems.

Assignment Sheets

During the day after each lecture we publish an assignment sheet with tasks related to the lecture under Materials. These usually consist of three written exercises and one programming exercise. Assignment sheets are to be completed by you in groups of 2 to 3 students and submitted via the CMS before the start of the next lecture. Afterwards, your submissions will be corrected by our team and you will receive points and feedback via the CMS. More information about the exact dates for the release and submission of the assignment sheets can be found in the Timetable.

Exam Admission

In order to be allowed to take part in the written exam and/or the re-exam, at least 50% of the possible points from the assignment sheets must be achieved. In addition, a maximum of 2 assignment sheets may be graded with 0 points.

Exam

At the end of the semester there will be both a written exam and a re-exam.

  • Exam: 27.07.23, 10:00 - 12:00
  • Re-exam: 15.09.23, 14:00 - 16:00

Grade

The better result of the written exam and the re-exam determines 100% of the final grade. In order to pass the lecture, at least 50% of the points must be achieved in the written exam or the re-exam.



Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators