LlamaIndex Newsletter 2024-04-16

Hello, LlamaIndex Family! 🦙

Welcome to another thrilling weekly update from LlamaGalaxy! We're excited to bring you a variety of outstanding updates, including the Chain of Abstraction LlamaPack, create-tsi, demos, guides, tutorials, and much more.

Before we delve into these updates, we have an exciting tutorial series on Agents and Tools for you to check out. Perfect for beginners, this series covers everything from advanced QA/RAG implementations to step-wise execution. By the end, you’ll have gained a deeper understanding of how to use agent reasoning with tool use to build simple applications. Check them out:

1. Overview
2. ReAct agents
3. Function Calling agents
4. Retrieval-Augmented agent
5. Controlling tool outputs
6. Agents with step-by-step execution

🤩 The highlights:

• Chain of Abstraction LlamaPack: Chain of Abstraction technique as llamapack a method enabling multi-step reasoning for enhanced tool use introduced by Silin Gao's team. LlamaPack, Tweet.
• Create-tsi Toolkit: Launched a toolkit for building full-stack RAG applications with customizable features like web crawling, local file indexing, and multilingual support, all hosted in EU data centers. Code, Tweet.
• Improved Agent Control: return_direct feature in tools allows direct output returns, reducing costs and enhancing response efficiency. Docs, Tweet.

✨ Feature Releases and Enhancements:

1. We have introduced the Chain of Abstraction Technique Developed by Silin Gao, and team as LlamaPack, this new method enables LLMs to generate multi-step reasoning chains for efficient sequence planning, enhancing tool use beyond single-shot functions. LlamaPack, Tweet.
2. We have launched create-tsi: A toolkit in collaboration with T-Systems and Marcus Schiesser to generate GDPR-compliant, full-stack AI applications via a CLI interface. Build enterprise-grade RAG bots with customizable features like web crawling, local file indexing, and multilingual support, all hosted in EU data centers. Code, Tweet.
3. We have introduced return_direct feature in tools that enhances agent controllability by allowing direct output returns as final responses. This optimizes for reduced latency and costs, and effectively halts the agent after crucial actions like booking confirmations or answering queries. Docs, Tweet.

🎥 Demos:

• RAG-enhanced MetaGPT: A robust multi-agent framework that features structured team dynamics for problem-solving, now supercharged with domain-specific knowledge from LlamaIndex modules. This framework supports diverse data inputs, sophisticated retrieval options, and efficient data management for enhanced agent performance.

🗺️ Guides:

• Guide to Building and Evaluating Advanced RAG by Hamza Gharbi for setting up a basic RAG pipeline, defining custom evaluation functions, and optimizing retrieval techniques.
• Paper by Prof. Markus J. Buehler: Using LLM-Generated Knowledge Graphs to Accelerate Biomaterials Discovery - This study showcases how a comprehensive knowledge graph from over 1000 scientific papers reveals novel insights and connections, driving innovation in biomaterials through art as inspiration. The KG construction was done with the help of LlamaIndex modules.
• Guide to Full-Stack RAG Application with AWS Bedrock: Set up Bedrock embeddings, use LlamaIndex for PDF retrieval, and build an interactive Streamlit interface, an ideal resource for enterprises starting with AWS services.
• Guide to Building a Lightweight ColBERT Retrieval Agent: Learn how to create an agent capable of advanced document retrieval and maintaining conversation memory, without the complexity of heavyweight agent frameworks.
• Guide to the Best RAG Techniques: 'ARAGOG' paper by Matous Eibich is a comprehensive evaluation survey exploring various RAG methods from classic vector databases to LlamaIndex's advanced techniques. Key findings highlight the effectiveness of HyDE, LLM reranking, and sentence window retrieval for improving precision and answer similarity.

✍️ Tutorials:

• Akash Mathur’s tutorial on Data Management in LlamaIndex: Featuring LlamaCloud and its open-source counterpart, this tutorial showcases efficient live data handling to minimize costs and latency in LLM applications.
• Leonie’s interactive tutorial to create an app that lets you converse with code from a GitHub repository.
• kingzzm’s tutorial on enhancing RAG Performance to overcome the issue of 'broken' context in RAG construction by dynamically creating contiguous chunks with auto-merging retrieval.
• Activeloop’s tutorial on Multimodal RAG for Pill Search teaches how to identify pills using images and text. This helps in identifying unknown pills, checking drug interactions and side effects, and confirming proper dosage amounts.
• Fanghua Yu's tutorial on using LlamaParse for Knowledge Graph Creation from Documents.

🎥 Webinars:

• Webinar with Tianjun Zhang and Shishir Patil, the two lead co-authors of RAFT, where they presented RAFT and discussed fine-tuning and RAG.