Software Project: Linguistic Interpretibility for Neural Models of Language

Thank you for submitting your proposals! These projects look quite exciting, and I look forward to seeing the results.

Our first check-in meeting will take place on Thursday, May 22. I would suggest 3 pm - please let me know if this time doesn't work for you and we can reschedule for another time that day. Here is the meeting link: https://teams.microsoft.com/l/meetup-join/19%3ameeting_NTRlZDkxYWEtMjE0Yy00ZmRiLWE5NjUtMzg0NzYzZmM1N2Ez%40thread.v2/0?context=%7b%22Tid%22%3a%2267610027-1ac3-49b6-8641-ccd83ce1b01f%22%2c%22Oid%22%3a%2208890f6f-a06a-4788-81bf-b1982132b1b4%22%7d

The goal of the meeting will be to discuss initial plans for the project and get feedback in a peer-to-peer setting. I expect that you will have started exploring the initial goal stated in your proposal. Please be prepared to briefly present your progress. This will be quite informal - no need for slides (unless you would find them helpful), but e.g. sharing your screen to display code, data, preliminary plots, etc. as needed would all be fine. The main purpose is to check in and provide support if needed; if you find yourself stuck on anything, bring it to the meeting and we'll discuss it. If I have more detailed feedback on your project plan, I will bring this to the meeting as well. In the meantime, feel free to email me with any questions!
 

Thanks to those of you who have enrolled in this software project! You should all have received an email from me by now, confirming your participation in this course.

The first milestone for course participation is the project proposal. This should be 3 pages in ACL format. I have extended the due date from May 1 to Monday May 5. Your proposal should clearly identify:

• The linguistic feature/s you want to investigate, with citation/s to relevant literature. 
  • This could include theoretical linguistic descriptions of the feature and/or computational models of this feature as appropriate.
  • You should also name the specific language/s you wish to focus on, and why.
• The proposed modeling approach, with relevant citations - see references on the course page. The scope of the project should be clearly specified. Some questions to consider:
  • Will you train your own model, or use pre-trained resources? 
  • What, if any, aspects of model performance are relevant to your analysis? For example, should the model pass some accuracy threshold on a particular task, in order to support your evaluation?
  • Which aspects of model architecture do you anticipate being relevant to your analysis? For example, when working with a Transformer model, certain linguistic features may be encoded in earlier or later layers (cf. Tenney et al., 2019, linked on the course page).
• The initial hypotheses you plan to investigate, and what you expect to find.
  • An especially sound proposal would also include some contingency plans, of how you would handle cases where things go wrong - this often happens in research! For instance, if you plan to train your own model, you may also want to sketch out an alternative approach to your hypotheses using pre-trained resources, in case you run into problems training the model.
  • You may also want to outline follow-up directions for investigation, based on what you expect to see from your initial hypotheses. For example, "I expect that the model will have a shared representation for Feature A in both Language X and Language Y; however, if I find that is not the case, then I will [do something else; e.g. explore Feature B because I think it's related, or identify edge cases, or...]."

The proposal will count for 20% of your grade, so do spend some time on it, and email me if you have questions. Grade allocations and timelines have been updated on the course page: https://lacoco-lab.github.io/courses/ling-interp-langmod/