LlamaIndex Newsletter 2024-11-19

Hello, Llama Followers! 🦙

Welcome to this week’s edition of the LlamaIndex newsletter! We’re excited to share updates on dynamic section retrieval, a new RAG technique for cohesive document retrieval, and the integration of ColPali for enhanced multimodal RAG results. This edition also features the launch of create-llama with a "Form Filler" agent for Typescript applications, and a guide to building multimedia research report generators combining text and images. Don’t miss these updates along with interesting use cases and tutorials from the community.

If you haven't explored LlamaCloud yet, make sure to sign up and get in touch with us to discuss your specific enterprise use case.

🤩 The highlights:

• Dynamic Section Retrieval Introduced: A new RAG technique fetches entire document sections cohesively with metadata and a two-pass retrieval process. Cookbook, Tweet.
• ColPali Integration for Multi-Modal RAG: ColPali as re-ranker combining Cohere’s multimodal embeddings with ColPali for precise text and image-based results. Cookbook, Tweet.
• Create-Llama v0.3.12 Launched: The new version includes a "Form Filler" agent, streamlining LlamaIndex Workflow integration for Typescript applications. Tweet.
• Multimedia Research Report Guide: A tutorial on generating structured reports combining text and images from complex documents using agentic RAG workflows. Cookbook, Tweet.

🗺️ LlamaCloud And LlamaParse:

• We have introduced dynamic section retrieval, a new RAG technique that enhances retrieval by ensuring entire sections of a document are fetched cohesively. This approach starts with simple page-level chunking, adds section metadata, and completes with a two-pass retrieval process to maintain contextual continuity. Cookbook, Tweet.
• Guide to create a multimedia research report generator that synthesizes insights from complex documents into structured reports combining text and images, utilizing the potential of structured outputs for agentic RAG and report generation workflows. Cookbook, Tweet.
• Guide to create a multi-agent workflow using LlamaCloud and OpenAI GPT-4o for generating structured financial reports from 10K documents, demonstrating advanced retrieval of text, tables, and diagrams and detailing the researcher and writer steps involved in producing the final structured analysis. Tweet.
• We have released a blog post and video detail the building blocks for advanced report generation, highlighting how structured outputs, advanced document processing, knowledge base integration, multi-agent workflows, and template systems can automate complex document creation, saving teams 10-15 hours per report. Video, Blogpost, Tweet.

✨ Framework:

• We have integrated ColPali as a re-ranker for building multimodal RAG, ensuring highly relevant results in both text and image modalities. We use Cohere’s multimodal embeddings for initial retrieval and Cohere, ColPali for re-ranking for text and images respectively before generating responses. Cookbook, Tweet.
• We have launched create-llama v0.3.12 with a "Form Filler" agent, simplifying the integration of LlamaIndex Workflows for Typescript applications. Tweet.
• We have launched a new "Ask AI" widget in our Python documentation, powered by RunLLM, featuring an agentic RAG system that provides accurate, up-to-date coding solutions directly in response to user queries. Tweet.
• We have launched day-0 integration with latest Mistral Pixtral model - Pixtral-large. Docs, Tweet.

💡 Use-case:

• PureML, utilizes LLMs with LlamaIndex, and Reflex to enhance ML dataset management through automated cleaning, intelligent feature creation, and data consolidation. Blogpost.
• PursuitGov uses LlamaParse to transform their B2G services, parsing 4 million pages in a weekend, boosting document accuracy, and enabling clients to discover opportunities in public sector data. Blogpost.
• RAGformation, a tool that automatically generates cloud configurations from natural language descriptions, complete with visual flow diagrams, pricing estimates, and customizable recommendations. Blogpost.

✍️ Community:

• Ravi Theja’s video tutorial on Multi-Modal RAG with ColPali as re-ranker.
• Lingzhen Chen’s tutorial on Building an Interactive UI for Llamaindex Workflows.
• Fermin Blanco’s tutorial on Structured Data Extraction from Unstructured Documents using Llamaindex.
• Tutorial on building an Agentic RAG Query Engine with NVIDIA NIM and LlamaIndex.
• Tutorial on building a Context Refinement Agent for RAG Systems.

🎙️Webinar:

• Join us in the Memgraph Community Call, where we’ll explore using LlamaIndex and Memgraph for powerful GraphRAG applications.
• Join us for a hands-on workshop on building local agentic RAG applications using open-source LLMs with AIMakerspace on November 27. Learn from experts Dr. Greg Loughnane and Chris Alexiuk about setting up an "on-prem" LLM app stack.