LlamaIndex Newsletter 2024-03-12

Salutations, LlamaIndex fans! 🦙

It's been another thrilling week in LlamaLand! With the release of Anthropic’s new models Claude-3 Opus, Sonnet, and Haiku, we have numerous tutorials, cookbooks, and updates to share with you.

A quick reminder: we are running our first ever user survey. It takes only 3 minutes and it helps us out a lot! Now let’s dive in.

🤩 The highlights:

1. LlamaParse JSON Mode: A new feature that transforms PDF content into structured data, simplifying RAG pipeline development for complex documents containing images, text and tables. Notebook, Tweet.
2. Hierarchical Code Splitting: Enhance code understanding with a novel technique that organizes large codebases into a hierarchical structure for improved navigation and task-solving. LlamaPack, Tweet.
3. Anthropic Cookbook Series: Learn to build various LLM applications with Claude 3, ranging from simple to complex, through detailed guides and tutorials. Cookbooks, Tweet.

✨ Feature Releases and Enhancements:

• We launched LlamaParse JSON Mode, a new feature that structures text and images from PDFs into a dict format. With the integration of multimodal models like claude-3 opus, it's now simpler to develop RAG pipelines for complex PDFs containing text, images, and tables. Notebook, Tweet.
• We launched a novel hierarchical code splitting technique to enhance RAG/agents for code comprehension, featuring CodeHierarchyNodeParser by ryanpeach. This method breaks down large code files into a hierarchical structure, enabling a knowledge graph-like approach for efficient code navigation and task-solving. LlamaPack, Tweet.
• We integrated with Videodb to run RAG over video streams using LlamaIndex. This tool allows you to upload, search, and stream videos based on spoken words or visual scenes, now available as a built-in retriever in LlamaIndex. Tweet.

🎥 Demos:

• Build an AI Browser Copilot: a project by Daniel Huynh that demonstrates how to create a browser agent using RAG, local embeddings, and Mixtral to execute browser tasks from a Colab notebook, showcased with a video on navigating HuggingFace datasets.
• RAG over your code: a project by Akshay on creating a local code assistant using LlamaIndex, MistralAI, and Streamlit to index and query GitHub repositories, offering a foundational guide for advanced code QA.
• https://nething.xyz/: a project by Raymond Weitekamp on generating production-ready 3D CAD models from text prompts. It uses LLM code generation to create commands to generate printable 3D objects from text prompts using LlamaIndex. Blog, Tweet.

🗺️ Guides:

• Guide to the Anthropic Cookbook Series: Create context-augmented LLM apps using Claude 3, from basic RAG to advanced agents, through six notebooks and four videos.
• Video guide exploring diverse applications of Claude-3 with LlamaIndex. tooling, covering Vanilla RAG, routing, sub-question query planning, structured data extraction, text-to-SQL, and agents—a perfect starter kit for Claude enthusiasts.

✍️ Tutorials:

• Tutorial by Ankush k Singal on building local LLM agents with Llama.cpp for step-wise execution and incorporating human feedback during execution.
• “RAG over Complex PDFs V2": a comprehensive tutorial by AI Makerspace on crafting advanced RAG pipelines for handling messy PDFs with LlamaParse and LlamaIndex, where naive RAG falls short.
• Tutorial on building a real-time RAG chatbot using Google Drive and Sharepoint by Anup Surendran and Berke Can Rizai.
• Step-by-step tutorial to improve the quality of your RAG application using JinaAI reranker, LlamaIndex, and MistralAI.

🎥 Webinars:

• Webinar with Parth Sarthi, lead author of RAPTOR - Tree-Structured Indexing and Retrieval.

📅 Events:

• We are hosting a RAG meetup in Paris on March 27th featuring talks on advanced RAG strategies, building a RAG CLI, and the significance of open-source RAG in business.