Data/ML/AI engineers, Data scientists, Software engineers interested in data processing, IT professionals looking to understand and apply RAG
- Programming Language: Python
- Topic: Real-time Retrieval Augmented Generation for Structured and Unstructured Data
- Target Geographic Region: North America
Retrieval Augmented Generation (RAG) is a technique that enhances the capabilities of Large Language Models (LLMs) by dynamically incorporating external information during the generation process. This approach combines the generative strengths of LLMs with the retrieval of relevant information from a large dataset, allowing for more informed, accurate, and contextually relevant outputs.
By enabling real time analytics capabilities within RAG systems, we can ensure users obtain the latest information.
In this workshop, we will focus on designing RAG pipelines through data flow pipeline design including Directed Graphs (DG), and the integration of real-time analytics. Through a blend of theoretical insights and hands-on sessions, attendees will learn how to set up efficient RAG pipelines that cater to the complexities of varied data types, driving enhanced decision-making and insights in their organizations. This workshop will use financial data as its use case, and will leverage a combination of structured and unstructured data such as stock and market prices as well as publicly available news. One of the challenges of financial data is how quickly it becomes irrelevant - both in terms of the market prices and any events around it.
Goals
- Gain proficiency in setting up and managing RAG pipelines for diverse data types.
- Leverage Bytewax for integrating real-time analytics into your data processing workflows.
- Unstructured TBD
- AI Co-Innovation Lab TBD
- Understand the nuances of working with both structured and unstructured data in a unified system.
- Develop the skills to architect, deploy, and optimize advanced RAG systems in real-world scenarios.
Pre-requisites
- Basic understanding of data structures and algorithms
- Familiarity with Python programming
- Basic knowledge of data processing and ETL concepts