AI4DH Lecture: Trends and Challenges in Artificial Intelligence

On Friday, 5 December, Dr Marko Robnik-Šikonja gave a presentation on trends and challenges in artificial intelligence as part of the beginner workshop series.

The lecture at the Institute of Contemporary History focused on how artificial intelligence works, deep learning and its weaknesses, large language models (LLMs), transformer applications, and new methods for training and using LLMs. Finally, he addressed the question of whether LLMs are close to achieving Artificial General Intelligence. In the concluding section, his lecture covered the EU guidelines for the responsible use of AI in research.

What are the new ways to train and use LLMs?

A recent development is in-context learning, where developers provide domain-specific data as context for a given problem. This enables the model to generate more relevant and useful responses. Another method is retrieval-augmented generation (RAG), which improves the reliability and accuracy of generative models by drawing on external, relevant data sources. Chain-of-thought reasoning is another approach, in which a problem is broken down into smaller, more manageable steps. The lecture also covered LLM agents, a relatively new approach that uses large language models to perform complex, multi-step tasks.

Participants were interested in several topics, including China’s growing role in the field of AI; how a model knows when to stop generating text; how we can determine the sources from which a model learns or retrieves information; whether LLMs can be used to scan old texts written in Gothic script; and what hallucination is and why it occurs.