Towards Safe and Controlled Large Language Models

Website •Slide •Code

9:00am – 12:00am (UTC/GMT+8, AWST) on Monday, 25 November 2024
Location: Level 3, RMIT University Building 80, Melbourne, VIC, Australia.

Large Language Models (LLMs) have significantly advanced natural language generation, yet they can produce dangerously flawed outputs, such as misinformation in medical advice or content that violates ethical standards. This tutorial explores these pressing issues and presents cutting-edge methods for creating safer, more controlled LLMs. Part I focuses on understanding hallucinations—instances where LLMs generate confidently incorrect or misleading information. It presents a taxonomy of hallucinations, and covers techniques for detecting them, including uncertainty assessment methods, score-based detection, and advanced model-based approaches.
Part II studies various strategies to mitigate hallucinations and enhance LLM reliability, such as reinforcement learning with human feedback to align models with human values, and direct preference optimization to fine-tune behaviour. It also examines prompt optimization methods to guide LLMs towards safer, more accurate responses without extensive retraining. Through practical examples and case studies, participants will learn essential tools and techniques to build LLMs that are not only powerful but also safe, reliable, and aligned with ethical standards.

This tutorial is primarily aimed at students and academics working with large language models (LLMs). It is also open to research engineers and industry professionals who need to apply LLMs safely and in a controlled manner in their work. While a basic familiarity with LLMs is expected, additional knowledge of deep learning and neural networks will be helpful. No special equipment is needed; however, attendees are encouraged to bring laptops for hands-on experimentation with the models.

Technical Understanding: Participants will develop a deeper understanding of the challenges of controlling and safely deploying LLMs, focusing on methods to reduce hallucinations and improve alignment.
Practical Skills: Attendees will gain hands-on experience in implementing various techniques for enhancing LLM reliability and safety, including practical demonstrations in prompt optimization, hallucination mitigation, and alignment training.

Dr. Hung Le is an Australian Research Council DECRA Fellow and a Research Lecturer at Deakin University, Australia. He is a senior member of the Applied Artificial Intelligence Institute (A2I2) where he currently supervises several PhD students in research areas focused on machine learning (ML) and reinforcement learning (RL). He specializes in deep learning and is dedicated to pioneering new agents equipped with artificial neural memory. His extensive work includes multi-modal, adaptive, and generative memory, efficient policy optimization, and memory-based reinforcement learning agents. With applications spanning health, dialogue agents, robotics, reinforcement learning, machine reasoning, and natural language processing, Dr. Le consistently publishes in premier ML/RL/AI conferences and journals, including ICLR, NeurIPS, ICML, AAAI, IJCAI, TMLR, KDD, NAACL, ECCV, and AAMAS. He earned his Bachelor of Engineering (Honors) from Hanoi University of Science and Technology and completed his PhD in Computer Science at Deakin University in 2015 and 2020, respectively.
Manh Nguyen is a first-year PhD student at A2I2, specializing in LLM Explainability. He earned his Bachelor of Electronics and Communications Engineering from Hanoi University of Science and Technology in 2017.