Date: September 11, 2024 (Wednesday)
Interactive Showcase: 2:30 – 5:30 pm
RAG Workshop: 3:30 – 5:30pm (pre-registration required)
Venue: Tam Wing Fan Innovation Wing One, HKU
Get ready for an exciting workshop featuring the latest student projects in the field of Generative AI. Our Student Research Assistants have been working hard on interesting projects that are sure to pique your interest. Come join us to have a first-hand experience and discover the possibilities of these projects!
All the three students research assistant projects will be featured at Makerspace A, with a designated booth for each project. An introductory PowerPoint presentation or short video clip will be prepared for each project and shown at their respective booth. Participants will have the opportunity to engage in hands-on experiences with all three projects. The brief descriptions of all three projects are shown below:
- AIHA (AI Historian Assistant): An AI-powered system designed to assist historians and general users in streamlining the retrieval and connection of scattered, multimodal historical sources they provide.
- CLIC-Chat 3.0: CLIC-Chat 3.0 extends our previous CLIC-Chat series by incorporating formal legal analysis and reasoning strategies.
- SWITCH (Social Work Interactive Training CHatbot): SWITCH aims to develop a chatbot to roleplay users of social services with different backgrounds, facilitating the training of social workers.
Date: September 11, 2024 (Wednesday)
Time: 2:30 – 5:30pm
Venue: Makerspace A, Tam Wing Fan Innovation Wing One, HKU
Join us for an engaging Generative AI workshop, focusing on Retrieval-Augmented Generation (RAG) systems and image retrieval for all levels of multimodal AI enthusiasts. Participants will grasp the foundational concepts of RAG and its multimodal extensions, exploring various methods and their practical applications.
Throughout the workshop, you will engage in theoretical discussions and hands-on activities, including text Optical Character Recognition (OCR), image extraction, and caption pairing with GPT-4o, to prepare your own datasets. You will also learn about vector search principles and embedding techniques, culminating in a complete pipeline demonstration comparing RAG to standard GPT. Join us to explore the exciting possibilities of multimodal AI, and elevate your skills to new heights!
Date: September 11, 2024 (Wednesday)
Time: 3:30- 5:30pm
Venue: Tam Wing Fan Innovation Wing One, HKU
Class size: Limited quotas are offered on first-come-first-served basis. No prior knowledge is required.
Eligibility: Priority to members of Innovation Wing! All HKU students are welcome to join.
Instructor: AI student research assistants of Innovation Wing
By the end of the workshop, students are expected to:
- Gain understanding of Retrieval-Augmented Generation (RAG).
- Get familiarize with Optical Character Recognition (OCR) and the process of creating text & image-caption dataset.
- Be able to perform vector search and embedding.
- Successfully implement multimodal RAG.
| Session | Duration | Details |
|---|---|---|
|
Introduction |
5 minutes |
Workshop Overview and Expectations |
|
Fundamentals and Essential Elements of RAG and Multimodal RAG |
10 minutes |
1. Introduction to Multimodal RAG 2. Introduction to HKU AI Historian Assistant (Inno Wing Student Research Assistant Project) |
|
Database Preparation |
30 minutes |
Participants will learn and perform the following tasks: 1. Text OCR 2. Image Extraction 3. Image-caption pairing using GPT-4o |
|
Vector Search and Embedding for Text and Image Retrieval |
30 minutes |
Participants will learn and perform the following tasks: 1. Introduction and Implementation of Vector Search through Embedding Models 2. Introduction to Algorithms: Cosine Similarity and BM25 3. Hands-on Exercise: Model Embedding |
|
Break |
5 minutes |
|
|
Context Generation through Multimodal RAG Pipeline
|
15 minutes |
1. Introduction to Prompt-engineering for Response Generation 2. Introduction to Complete Multimodal RAG System Pipeline 3. Hands-on Exercise: Implementation of Multimodal RAG; Multimodal RAG comparison using GPT |
|
Conclusion |
15 minutes |
1. Workshop Summary and Key Takeaways 2. Discussion: Pros and Cons of RAG; Comparison with Alternative Approaches |
|
Q&A Session |
5 minutes |
|
