LlamaIndex Newsletter 2023–11–21

Hello Llama Fam 🦙

What an amazing week we’ve had! We’re excited to share that, according to the Retool State of AI 2023 survey, 1 in 12 respondents are now using LlamaIndex. We’re grateful for all your support.

If you have a fascinating project or video you’d like to share, we’d love to see it! Feel free to send it to us at news@llamaindex.ai. And remember to subscribe to our newsletter on our website to stay in the loop. We can’t wait to connect with you there!

🤩 First, the highlights:

1. LlamaIndex 0.9 Release: we introduced LlamaIndex version 0.9 featuring streamlined data handling with a new IngestionPipeline, automated caching, improved text processing interfaces, tokenizer updates, PyPi packaging enhancements, consistent import paths, and a beta version of MultiModal RAG Modules. Blog post, Tweet.
2. Multi-Modal Evaluation Tools: we launched multi-modal evaluation with the introduction of MultiModalRelevancyEvaluator and MultiModalFaithfulnessEvaluator, plus a guide for their application in multi-modal settings. Blog post, Tweet.
3. create-llama CLI Tool: we unveiled create-llama, a versatile CLI tool for building full-stack LLM apps with options like FastAPI, ExpressJS, and Next.js for backends and a Next.js frontend with Vercel AI SDK components. Blog post, Tweet.
4. Cohere Reranker Fine-Tuning: we enhanced RAG pipeline retrieval performance with the fine-tuning of the Cohere reranker. Blog post, Tweet.

Coming up this week: we have a YouTube live event in partnership with AI Makerspace exploring the potential of LlamaIndex to handle complex PDFs with tables, charts and more. Register for free!

✨ Feature Releases and Enhancements:

• We introduced the LlamaIndex 0.9 version with updates on streamlined data handling with new IngestionPipeline, automated caching, improved interfaces for text processing, tokenizer updates, enhanced PyPi packaging, consistent import paths, and a beta of MultiModal RAG Modules for text and image integration. Blog post, Tweet.
• We introduced multi-modal evaluation which includes MultiModalRelevancyEvaluator and MultiModalFaithfulnessEvaluator, and a guide on using them in multi-modal applications. Blog post, Tweet.
• We introduced create-llama, a CLI tool for easily building full-stack LLM apps, offering choices like FastAPI, ExpressJS, and Next.js backends with Llama Index, and a Next.js frontend with Vercel AI SDK components, enabling extensive customization for AI engineers. Blog post, Tweet.
• We introduced fine-tuning of the cohere reranker to improve retrieval performance in the RAG pipeline. Blog post, Tweet.

Integrations:

• We integrated with Chroma’s multi-modal collections which allows for indexing both text and images in a single collection, enhancing RAG pipelines by combining text and image information for use with multi-modal models like GPT-4V, LLaVa, and Fuyu. Docs, Tweet.

🗺️ Guides:

• Guide on Multi-Modal Retrieval using GPT text embedding and CLIP image embedding for Wikipedia Articles.
• Guide on LlamaIndex by Nanonets covering over 12 key areas such as data management, indexing/storage, querying with top-k RAG, structured outputs, chat functionalities with memory, and agent development incorporating tool use.
• Guide on using Ingestion pipeline focusing on showcasing experiments on chunk overlaps and the use of metadata extractors, including title, summary, and other elements.
• Guide on using Perplexity API with LlamaIndex by Vishhvak.
• Guide on using Fleet Context to download the embeddings for LlamaIndex’s documentation and build a hybrid dense/sparse vector retrieval engine on top of it.
• Guide on building a full-stack financial analysis bot using create-llama and Llama Index's RAG, capable of querying text and tables across SEC filings.

✍️ Tutorials:

• Wenqi Glantz made a tutorial on LLaVA vs. GPT-4V Amidst Snow Geese Migration.
• Glenn Parham’s cookbook on LlamaIndex, hosted in the Department of Defense’s official repository, showcases methods for applying RAG on unclassified DoD policy documents.
• Sudarshan Koirala made a tutorial on Using Perplexity API with LlamaIndex.
• Ravi Theja analysis on GPT4-V Experiments with General, Specific questions and Chain Of Thought prompting(COT) techniques

🎥 Webinars:

• Check out our CEO — Jerry Liu’s talk on Building Production-Ready RAG Applications at AI.engineer Summit.