News

Project Update and Clarifications

Written on 21.06.24 (last change on 21.06.24) by Luca Gretscher

Dear students,

due to an issue with task T2c (see forum posts), we have just uploaded a new version of the project under Materials -> IMDb (Part 3). Note that only the project description and the test file `famesocialnetwork/tests.py` differ slightly from the original version. So do not worry about… Read more

Dear students,

due to an issue with task T2c (see forum posts), we have just uploaded a new version of the project under Materials -> IMDb (Part 3). Note that only the project description and the test file `famesocialnetwork/tests.py` differ slightly from the original version. So do not worry about merging your current progress into the updated version, just replace the test file! According to the updated project description, you do not need to change the MyLoginView class in any way (this makes the task actually easier), thus, we will also replace the corresponding file with a fresh version when determining your final test results.

Additionally, here are some clarifications:
    •    The bonus points of tasks T8 and T9 are only for the project, not for the final grade of the entire course.
    •    The code review is mandatory for all team members regardless of the number of points achieved through the unit/end-to-end tests.
    •    Only register your team for a single code review slot, otherwise not enough slots will be available for other teams. Keep in mind that only one team member registers for the entire team.

Kind regards,
Your BDE Team

Project Release

Written on 20.06.24 (last change on 20.06.24) by Luca Gretscher

Dear students,

we have just uploaded the project under Materials -> IMDb (Part 3). A detailed project description including instructions for the submission can be found in the project.pdf file in the root directory of the project. The submission deadline for the project is Sunday, 07.07.24,… Read more

Dear students,

we have just uploaded the project under Materials -> IMDb (Part 3). A detailed project description including instructions for the submission can be found in the project.pdf file in the root directory of the project. The submission deadline for the project is Sunday, 07.07.24, 23:59!

Additionally, each group must participate in a code review after the project submission. We offer eight code review slots:
    •    Slot 1: Wednesday 10.07.24, 10:00 - 12:00
    •    Slot 2: Wednesday 10.07.24, 12:00 - 14:00
    •    Slot 3: Thursday     11.07.24, 08:00 - 10:00
    •    Slot 4: Thursday     11.07.24, 12:00 - 14:00
    •    Slot 5: Thursday     11.07.24, 14:00 - 16:00
    •    Slot 6: Thursday     11.07.24, 16:00 - 18:00
    •    Slot 7: Friday          12.07.24, 08:00 - 10:00
    •    Slot 8: Friday          12.07.24, 10:00 - 12:00

More detailed information about the code reviews will be published at the end of the project, but keep in mind that every team members must know and understand all code. Team members who are not able to explain parts of the code do not get points for the project.

Please register for the code review on your personal status page by Sunday, 07.07.24, 23:59! Note that only one team member is allowed to register for the entire team, so please coordinate internally! Registration is on a first-come-first-served basis.

Kind regards,
Your BDE Team

Tutorials on 25.06.2024

Written on 19.06.24 by Luca Gretscher

Dear students,

due to the talk by Richard Hipp next Tuesday, we will adapt tutorials 5&6. If you want to attend the talk, you are welcome to attend another tutorial slot next week. However, we will still offer a single tutorial in the regular time slot in E1 1, 3.06 for those who do not want to… Read more

Dear students,

due to the talk by Richard Hipp next Tuesday, we will adapt tutorials 5&6. If you want to attend the talk, you are welcome to attend another tutorial slot next week. However, we will still offer a single tutorial in the regular time slot in E1 1, 3.06 for those who do not want to attend the talk.

Kind regards,
Your BDE Team

Reminder: Team Groupings for Project

Written on 19.06.24 by Luca Gretscher

Dear students,

we would like to remind you that the deadline for creating or joining a team for the project submission is tomorrow morning at 12am. If you are still looking for a team, you can use the forum.

If you correctly joined a team, only one team member has to upload your submission.… Read more

Dear students,

we would like to remind you that the deadline for creating or joining a team for the project submission is tomorrow morning at 12am. If you are still looking for a team, you can use the forum.

If you correctly joined a team, only one team member has to upload your submission. Please make sure that every team member is able to see the uploaded submission on their personal status page. Only then, all team members will receive the points.

Please note, that only submissions in groups of 3 to 4 students are allowed. Submissions of smaller groups will not be graded.

Kind regards,
Your BDE Team

Talk by Richard Hipp, Creator of SQLite, Tuesday, June 25th, 12:15-14:00, GHH

Written on 18.06.24 by Jens Dittrich

Title:
How SQLite Works

Speaker:
Richard Hipp

When:
Tuesday, June 25th, 12:15-14:00

Where:
GHH

Abstract:
If you measure success by the number of deployments, SQLite dominates the
database world. There are more instances of SQLite running today than all
other database engines… Read more

Title:
How SQLite Works

Speaker:
Richard Hipp

When:
Tuesday, June 25th, 12:15-14:00

Where:
GHH

Abstract:
If you measure success by the number of deployments, SQLite dominates the
database world. There are more instances of SQLite running today than all
other database engines combined.  Why is this?  How did SQLite come to be
so widely used?  What makes SQLite different?

This talk attempts to answer those questions by reviewing the unusual
design and implementation choices in SQLite, and showing how those
choices make SQLite well suited for mobile phones, appliances,
applications, and the "internet of things" where SQLite is often found.
The talk will delve into SQLite's use of bytecode, the techniques used
to achieve ACID power-safe transactions, and the query planning
algorithms.

This talk is targeted toward listeners who are familiar with
other database implementations, but it also accessible to people with
only a general computer programming background and who just want to
know more about how an SQL database engine works behind the scenes.


Short Bio:
Richard Hipp is an entrepreneur based in Charlotte, NC, USA, and the founder
and lead developer for SQLite as well as several other open-source projects.
He has been an invited speaker at universities around the world, including
MIT, Harvard, CMU, NYU, IIT Kharagpur, Politechnika GdaƄska, and others.

Richard never intended to become a "database guy".  His PhD dissertation
(Duke University, 1992) was in computational linguistics, and his
undergraduate studies were in electrical engineering and digital signal
processing.  Because he did not come up inside the database community,
Richard brings a different perspective to database design, implementation,
and usage.

When he isn't sitting at the keyboard writing code (his usual habitat),
Richard can often be found out on long-distance runs or cooking vegetarian
meals with his wife, Ginger.

Host:
Jens Dittrich

 

Assignment 06

Written on 13.06.24 by Luca Gretscher

Dear students,

we have just uploaded Assignment 06. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

Team Groupings for Project

Written on 07.06.24 by Luca Gretscher

Dear students,

the programming project will start in about two weeks. Analogously to the assignments, we allow submitting the project in teams of 3 to 4 students. Please note, that submissions of smaller groups will not be graded and thus the exam admission cannot be achieved. It is permitted to… Read more

Dear students,

the programming project will start in about two weeks. Analogously to the assignments, we allow submitting the project in teams of 3 to 4 students. Please note, that submissions of smaller groups will not be graded and thus the exam admission cannot be achieved. It is permitted to form teams different from those for the assignments, however, we recommend submitting the project in the same team. In any case, you have to additionally register your team!

To register your team, you have to use the Team Groupings functionality of the CMS. This means for you that a team member creates a group in the CMS on the personal status page and invites all other team members to the group. You can create your team until Thursday 20.06.24 at 12:00am, after that it is not possible to change the team.

If you haven't found a team yet, you can use our forum to find one.

Kind regards,
Your BDE Team

No Tutorials on 10./11.06.2024

Written on 07.06.24 (last change on 07.06.24) by Luca Gretscher

Dear students,

due to the public holiday on the penultimate Thursday, we do not have an assignment sheet to discuss next week. Therefore, all tutorials next week will be cancelled.

The office hour on Wednesday will take place as usual.

Kind regards,
Your BDE Team

Assignment 05

Written on 06.06.24 by Luca Gretscher

Dear students,

we have just uploaded Assignment 05. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

Tutorial Room Change

Written on 31.05.24 by Luca Gretscher

Dear students,

please note, that the seminar room 2.06 in building E1 1 is otherwise occupied next week. Therefore, Tutorial 6 (Tuesday 12:15) will take place in E1 1, 1.06.

Kind regards,
Your BDE Team

Merging of Tutorials

Written on 31.05.24 by Luca Gretscher

Dear students,

since the number of participants in the tutorials is continuously decreasing, we have decided to merge some of the tutorials that take place at the same time. Concretely, we will be merging the tutorials 1 & 2, the tutorials 3 & 4, and the tutorials 7 & 8 from now on. Here is an… Read more

Dear students,

since the number of participants in the tutorials is continuously decreasing, we have decided to merge some of the tutorials that take place at the same time. Concretely, we will be merging the tutorials 1 & 2, the tutorials 3 & 4, and the tutorials 7 & 8 from now on. Here is an overview of the tutorial slots starting next week.

    •    Tutorial 1 & 2: Monday, 10:15 in E1 1, 3.06
    •    Tutorial 3 & 4: Monday, 12:15 in E1 1, 3.06
    •    Tutorial 5: Tuesday, 12:15 in E1 1, 3.06
    •    Tutorial 6: Tuesday, 12:15 in E1 1, 2.06
    •    Tutorial 7 & 8: Tuesday, 14:15 in E1 1, 3.06

You can find a concise overview in our Timetable as well.

Kind regards,
Your BDE Team

Assignment 04

Written on 23.05.24 by Luca Gretscher

Dear students,

we have just uploaded Assignment 04. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

Exam Registration in Flex Now

Written on 23.05.24 (last change on 23.05.24) by Luca Gretscher

Dear students,

in case your course of study requires a registration in Flex Now, e.g. Wirtschaftsinformatik and related subjects, please have a look at this forum post. As far as we know, the deadline ends today at 3pm.

Kind regards,
Your BDE Team

No Tutorials on 21.05.2024

Written on 20.05.24 (last change on 07.06.24) by Luca Gretscher

Dear students,

due to the public holiday on the penultimate Thursday, we do not have an assignment sheet to discuss this week. Therefore, all tutorials tomorrow will be cancelled.

The office hour on Wednesday will take place as usual.

Kind regards,
Your BDE Team

Assignment 03

Written on 16.05.24 by Luca Gretscher

Dear students,

we have just uploaded Assignment 03. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

no lecture today

Written on 16.05.24 by Jens Dittrich

Dear students,

I have become sick. Therefore, I cannot give the lecture today. For this week, we will fallback to the video recordings from last year. See the links in the materials section.

Best regards,

Jens Dittrich

Merging of Tutorial 9

Written on 10.05.24 by Luca Gretscher

Dear students,

since the number of participants in the tutorials is lower than expected, we have decided to merge tutorial 9 with tutorials 5 & 6 that take place at the same time. Therefore, we reassigned every student of tutorial 9 to one of the other two tutorials. Please check on your Personal… Read more

Dear students,

since the number of participants in the tutorials is lower than expected, we have decided to merge tutorial 9 with tutorials 5 & 6 that take place at the same time. Therefore, we reassigned every student of tutorial 9 to one of the other two tutorials. Please check on your Personal Status page where your new tutorial takes place.

Kind regards,
Your BDE Team

Update: Team Groupings

Written on 03.05.24 by Luca Gretscher

Dear students,

we recognized that a few students submitted in groups which did not meet the requirement of having at least 3 team members. Thus, these submission unfortunately will not be graded. There were also some single member teams that did not submit.

However, as we want to give you a fair… Read more

Dear students,

we recognized that a few students submitted in groups which did not meet the requirement of having at least 3 team members. Thus, these submission unfortunately will not be graded. There were also some single member teams that did not submit.

However, as we want to give you a fair chance for submitting the next assignment, we would like to offer the following possibilities to those of you that are still looking for a team. Please choose one of the two options.

  • You can search a team by yourself. There are still open threads in the forum for that. Write an email to Luca Gretscher until Monday, 06.05.2024, 10am and state the matriculation numbers of all your team members. Please also state whether the team was already created and some students want to join or whether a new team should be created.
  • If you do not find a team but want to submit the assignments anyway, please also write an email to Luca Gretscher until Monday, 06.05.2024, 10am and state your matriculation number. We will then create new teams for those students and randomly assign you to one of them.

Please note, that still only submissions in groups of 3 to 4 students are allowed. Submissions of smaller groups will not be graded.

Kind regards,
Your BDE Team

Assignment 02

Written on 02.05.24 by Luca Gretscher

Dear students,

we have just uploaded Assignment 02. Instructions for the submission can be found at the end of the document.

Kind regards,
Your BDE Team

Reminder: Team Groupings

Written on 01.05.24 by Luca Gretscher

Dear students,

we would like to remind you that the deadline for creating or joining a team for the assignment submissions is tomorrow morning at 10am. If you are still looking for a team, you can use the forum. There are still open threads.

If you correctly joined a team, only one team member… Read more

Dear students,

we would like to remind you that the deadline for creating or joining a team for the assignment submissions is tomorrow morning at 10am. If you are still looking for a team, you can use the forum. There are still open threads.

If you correctly joined a team, only one team member has to upload your submission. Please make sure that every team member is able to see the uploaded submission on their personal status page. Only then, all team members will receive the points.

Please note, that only submissions in groups of 3 to 4 students are allowed. Submissions of smaller groups will not be graded.

Kind regards,
Your BDE Team

Tutorial Assignment

Written on 26.04.24 by Luca Gretscher

Dear students,

we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start next week. More information can be found in the Timetable.

If you were assigned to Tutorial 9 (Monday, 00:00 No Tutorial) but still want to… Read more

Dear students,

we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start next week. More information can be found in the Timetable.

If you were assigned to Tutorial 9 (Monday, 00:00 No Tutorial) but still want to attend a tutorial at some point, you are welcome to attend the tutorial on Tuesdays 12:15 - 13:45 in the Günter-Hotz lecture hall.

Please bring your laptop to the tutorials since we will do some coding exercises (at least next week). Additionally, we will upload the tutorial slides before each tutorial.

Kind regards,
Your BDE Team

Assignment 01 & Team Groupings

Written on 25.04.24 by Luca Gretscher

Dear students,

we have just uploaded Assignment 01. Instructions for the submission can be found at the end of the document.

Please note that the submission is only allowed in teams of 3 to 4 students. Furthermore, we ask that you try to form teams of exactly 4 students to reduce the number of… Read more

Dear students,

we have just uploaded Assignment 01. Instructions for the submission can be found at the end of the document.

Please note that the submission is only allowed in teams of 3 to 4 students. Furthermore, we ask that you try to form teams of exactly 4 students to reduce the number of submissions.

To register your team, you have to use the Team Groupings functionality of the CMS. This means for you that a team member creates a group in the CMS on the personal status page and invites all other team members to the group. You can create your team until Thursday 02.05.24 at 10:00am, after that it is not possible to change the team. If you need to change your team during the semester, please contact your tutor.

If you haven't found a team yet, you can use our forum to find one.

Kind regards,
Your BDE Team

Reminder: Tutorial Preferences

Written on 25.04.24 by Luca Gretscher

Dear students,

we would like to remind you that you can change your preferences for the tutorials only until tonight.

We offer nine regular tutorials at the following times:
    •    Slot 1: Monday, 00:00 No Tutorial: This is a dummy tutorial! Please give preference to this tutorial if you do… Read more

Dear students,

we would like to remind you that you can change your preferences for the tutorials only until tonight.

We offer nine regular tutorials at the following times:
    •    Slot 1: Monday, 00:00 No Tutorial: This is a dummy tutorial! Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices.
    •    Slot 2: Monday, 10:15 Tutorial: Two regular tutorials at this time slot in E1.1 R2.06 and E1.1 R3.06
    •    Slot 3: Monday, 12:15 Tutorial: Two regular tutorials at this time slot in E1.1 R2.06 and E1.1 R3.06
    •    Slot 4: Tuesday, 12:15 Tutorial: Three regular tutorials at this time slot in E1.1 R2.06, E1.1 R3.06, and E2.2 0.01 (GHH)
    •    Slot 5: Tuesday, 14:15 Tutorial: Two regular tutorials at this time slot in E1.1 R2.06 and E1.1 R3.06

In the tutorials, we will in general briefly discuss the solution of the previous assignment sheet. However, the main focus will be a recap of the lecture material by providing additional exercises.
Please note, that both the solutions of the assignment sheets as well as the tutorial slides including solutions will be timely uploaded to our Materials.

Because of this, experience has shown that some students will not attend the tutorials in person. Therefore, we also created an additional (pseudo) tutorial (Slot 1: Monday, 00:00). Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices. With this, we can allow students that plan to attend, to more likely get their preferred choice. Furthermore, we can better assess how many in-person tutorials we really need. Of course, you can always attend a tutorial if you change your mind.

Please submit your tutorial preferences on your personal status page by 23:59 today!

Kind regards,
Your BDE Team

Vagrant & Docker

Written on 24.04.24 by Luca Gretscher

Dear students,

in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine using Vagrant that has all the required software and… Read more

Dear students,

in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine using Vagrant that has all the required software and systems already installed. You can find detailed instructions at the following Github repository. The link is also available under Information > Vagrant. We also provide a small FAQ.

As there are problems with Vagrant on Apple Silicon chips, we also created a Docker image which can be used instead of the Vagrant VM. You can find detailed instructions at the following Github repository. The link is also available under Information > Docker.

We spent a lot of time preparing the tools and the instructions. Please read them carefully before asking questions.

Please note, that you are also able to run most of the notebooks by just having a Python installation with the required packages.

If you still have problems with the installation, you can always ask a question in the forum (bde24 > Technical Problems).

Kind regards,
Your BDE Team

Show all

Big Data Engineering

This course covers the fundamentals of data management and query processing.

Learning objectives of this course are:
  • Learn basic techniques in "Big Data Engineering" conceptually.
  • Learn to apply basic techniques in "Big Data Engineering".
  • Help you not to reinvent the wheel later.
  • Learn to map new problems to existing problems and solve them using established techniques.
  • Raise awareness for problems of important applications: Privacy, deanonymization, ethical issues.
  • Raise awareness for solutions of important applications: Effort, performance, robustness, extensibility, maintainability.

Lecture

The lecture takes place every Thursday at 10:15 in building E2 2 - lecture hall 0.01 (Günter-Hotz-Hörsaal). The recording of the lecture will be uploaded to Youtube afterwards. The lecture was awarded with two busy beaver teaching awards (2011, 2021) and two busy beaver honorable mentions (2022, 2023).

Tutorials

On Mondays and Tuesdays there will be tutorials. In the tutorials there will be both exercises and the possibility to ask questions regarding the assignment sheets and the lecture. Please indicate your preferred tutorial timeslots on your personal status page by 23:59 on 25.04.2024. We will then assign you to your tutorial. If you are unable to attend on your assigned date at some point, you are welcome to attend another tutorial. The tutorials will start in the week of 29.04.2024.

Office Hours

In addition to the tutorials, there will be an Office Hour on Wednesdays from 14:15 to 15:45 in the seminar room E1.1 R3.06. In the Office Hour you can ask questions about the lectures and assignment sheets, as well as get help with technical problems.

Assignment Sheets

During the day after each lecture we publish an assignment sheet with tasks related to the lecture under Materials. These usually consist of three written exercises and one programming exercise. Assignment sheets are to be completed by you in groups of 3-4 students and submitted via the CMS before the start of the next lecture. Afterwards, your submissions will be corrected by our team and you will receive points and feedback via the CMS. More information about the exact dates for the release and submission of the assignment sheets can be found in the Timetable.

Project

Assignment sheets 7 and 8 will be replaced by a small project for which you have to develop a toy backend server for a social network. The project is to be completed by you in groups of 3-4 students (not necessarily the same group as for the assignments). Your points will be determined by functional tests (50%) and the quality of your code together with the outcome of a short code review (50%). Note that all team members must know and understand all code. Team members who are not able to explain parts of the code do not get points for the project.

Exam Admission

In order to be allowed to take part in the written exam and/or the re-exam, at least 50% of the possible points from both the assignment sheets and the project must be achieved each. In addition, a maximum of 2 assignment sheets may be graded with 0 points.

Exam

At the end of the semester there will be both a written exam and a re-exam.

  • Exam: 01.08.24, 10:00 - 12:00
  • Re-exam: 26.09.24, 14:00 - 16:00

Grade

The better result of the written exam and the re-exam determines 100% of the final grade. In order to pass the lecture, at least 50% of the points must be achieved in the written exam or the re-exam.

Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators.