News
Transformers Material
Written on 23.01.2025 19:40 by Michael Hahn
Hi all,
here is some background material on transformers:
- The authoritative reference is Vaswani et al 2017 https://arxiv.org/abs/1706.03762
- The Induction Head construction we discussed on Monday originates in this influential blog post: https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
- The First function we discussed on Monday is considered in Chiang & Cholak 2022: https://aclanthology.org/2022.acl-long.527/
- Tomorrow, I'd like to discuss the PARITY function and State Space Models. Relevant references are: https://arxiv.org/abs/2402.09963 , https://neurips.cc/virtual/2024/poster/94264 , https://arxiv.org/abs/2411.12537
Also, please evaluate the class, if you have not yet done so. The link is: https://qualis.uni-saarland.de/eva/?l=151774&p=87s328 It will go offline next week.
See you tomorrow,
Michael