News

Course evaluations

Written on 16.01.24 by Kate McCurdy

If you would like to give feedback on the seminar, you can fill out this evaluation form, which should take about 5 minutes: https://qualis.uni-saarland.de/eva/?l=145220&p=fn4p1t

I'd appreciate your perspectives on what might be going well and what could be improved, cheers!

Seminar schedule update

Written on 03.01.24 by Kate McCurdy

Happy 2024! A quick update on the course schedule: due to changes in the presentation schedule, our last seminar session will be held on January 30, rather than February 6 as originally planned. As always, you can find the updated schedule and readings on the course website.

I also failed to inform… Read more

Happy 2024! A quick update on the course schedule: due to changes in the presentation schedule, our last seminar session will be held on January 30, rather than February 6 as originally planned. As always, you can find the updated schedule and readings on the course website.

I also failed to inform you all earlier about the weekly questions. Firstly -- as everyone has various things come up over the course of a semester, I will plan to drop the lowest mark on weekly questions from each student. This means everyone gets at least one "freebie:" if you miss question submission for one week, it won't count against you.

Secondly, I meant to excuse the question-posting requirement for this week's class, given the holiday the day before; however, I didn't send the message out in time. This means: if you didn't submit a question this week, it's not problem, you still get a "freebie;" and for those of you who *did* submit a question this week for the January 2nd session, I'll count it as extra credit.

Feel free to email me with any questions, and wishing you all a fine start to the new year!

Project proposals: discussion next week

Written on 14.12.23 by Kate McCurdy

For next week's session on Dec. 19th, students who are planning to submit a term paper for this course will present a 5 minute project proposal for feedback. As described on the course website, term papers should focus on prompting an LLM to probe some ability relevant to cognition. We haven't… Read more

For next week's session on Dec. 19th, students who are planning to submit a term paper for this course will present a 5 minute project proposal for feedback. As described on the course website, term papers should focus on prompting an LLM to probe some ability relevant to cognition. We haven't discussed papers using prompting methods yet in the seminar, so it's understandable if you're not sure to start, but we will discuss some such papers after the break; you might get some inspiration by looking at the experimental design used by Hu et al. 2023, Kosinski 2022, Ullman 2023, or Binz and Schulz 2023.

Please email me before Tuesday to let me know if you are planning to present a proposal, so I can make sure we have enough time to get to everyone on the 19th. If you have presentation materials (e.g. 1-2 slides) and you don't want to bring your own laptop, feel free to email me your slides or other material until 3 pm on Tuesday, and I can share them from my laptop. I would recommend emailing me your materials so we don't need to waste time switching machines in class.

Your proposal is mainly to present your core research question and high-level experiment plan to get feedback from myself and the group, so that you have time to refine and improve your plan before putting too much effort into it. As such, the proposal discussion is purely for your benefit; you will not be graded or evaluated for this aspect of the project.

The main points for you to clarify are:

  1. The research question. What problem do you aim to address?
  2. The motivation. What will this project tell us about cognition?
    Remember, this seminar is focused on language models as cognitive models - your project should not simply focus on performance, but address a question relevant to cognitive science.
  3. The relevant background. Your project will describe, and test, whether a language model shows a particular behavior. What does the literature say about this question? Do we observe or expect this behavior from humans?
  4. The hypothesis. What behavior do you expect to see from the language model?
  5. The methodology. What experimental design will you use to evaluate your hypothesis? Will you adapt an existing benchmark of prompts to your needs, or will you develop your own?
    In either case, please include one or two example prompts from each relevant condition - this will make it significantly easier for your audience to understand your proposed design. If you are presenting slides, you may or may not choose to include all of the above points, but make sure you have some example prompts on your slides.

Ideally, each of these points can be addressed in one or two sentences, giving us time for discussion and feedback after each presentation. If you have additional questions, you can let me know by email; I might be able to address specific questions, although for general questions I will most likely refer you back to the advice given here.

Slides upload + recent wug research

Written on 15.11.23 by Kate McCurdy

Thanks to everyone for a lively presentation and discussion yesterday!

To make the slides accessible to everyone, I'll request that presenters post them on the relevant thread on the Forum. You should be able to upload a file, or post a link to somewhere other hosting location if that's preferable.… Read more

Thanks to everyone for a lively presentation and discussion yesterday!

To make the slides accessible to everyone, I'll request that presenters post them on the relevant thread on the Forum. You should be able to upload a file, or post a link to somewhere other hosting location if that's preferable. Contact me if there are any issues with presentation uploads.

And for any who are interested, here's a link to the new research I mentioned yesterday on wug-testing ChatGPT.

Question reminder + Presentation advice

Written on 13.11.23 by Kate McCurdy

Hi all! Two things:

1) a quick reminder to submit your question on the forum by noon today, so that presenters have time to review them before tomorrow's seminar.

2) this link has some advice from psychologist Paul Bloom on how to give good presentations. Not all of it will apply in our seminar,… Read more

Hi all! Two things:

1) a quick reminder to submit your question on the forum by noon today, so that presenters have time to review them before tomorrow's seminar.

2) this link has some advice from psychologist Paul Bloom on how to give good presentations. Not all of it will apply in our seminar, but I'd particularly emphasize points 5 (simplify, omit unnecessary detail) and 8 (keep focus on the big picture).

Bis morgen! -Kate

Today's session moved to next week

Written on 31.10.23 by Michael Hahn

Hi everyone,

 

Due to illness, today's session is postponed to next Tuesday, November 7.

 

You can find the slides here:

https://lacoco-lab.github.io/courses/langmod-cogmod-2023/session2.pdf

 

We will finalize the assignment of students to slots soon.

 

Best… Read more

Hi everyone,

 

Due to illness, today's session is postponed to next Tuesday, November 7.

 

You can find the slides here:

https://lacoco-lab.github.io/courses/langmod-cogmod-2023/session2.pdf

 

We will finalize the assignment of students to slots soon.

 

Best wishes,

Michael

Please let me know your preferences!

Written on 27.10.23 by Michael Hahn

Resending this info as more people have registered!


Hi everyone,
 
Thank you for registering for the course!
 
Given the number of registered students, we expect that each presentation will be done jointly by two students. In fact, we now have a few more students than we can… Read more
Resending this info as more people have registered!


Hi everyone,
 
Thank you for registering for the course!
 
Given the number of registered students, we expect that each presentation will be done jointly by two students. In fact, we now have a few more students than we can accommodate. If needed, we will prioritize students in such a way that every topic in the syllabus can be covered. Final decisions will be made after the Tuesday session next week.
 
Please send me an email with the following content by next Monday, October 30:
 
1. Your name
 
2. Your top four choices among the topics in the syllabus (starting with the most desired one)
 
3. If you want to present together with any particular student, let me know. No guarantee, however, that we'll be able to make that work.
 
4. If applicable, a reason why you think you should be prioritized (e.g. it's highly relevant to your research).

Finally, if anyone you know wants to take the course but hasn't registered in CMS, please kindly tell them to do so and to send me the above information.
 
Thank you!

See you next Tuesday,
Michael
 
 
Show all

Language Models as Cognitive Models

 

See https://lacoco-lab.github.io/courses/langmod-cogmod-2023/ for the syllabus and further information.

Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators.