Understanding large language models for digital humanities

This seminar provides the foundational knowledge for the next seminar, Adapting and Fine-tuning LLMs – a Hands-on Approach for DH.

Large language models are changing the way we write, read, and do intellectual jobs. We present the working of the transformer architecture of neural networks and focus on the decoder models, which are used in generative models, such as ChatGPT. Explaining their construction, pretraining, instruction following, preference alignment and fine tuning, we give necessary background to understand their behaviour. Based on this, we explain prompting strategies, such as in-context learning, and chain of thought reasoning.