LlamaIndex Newsletter 2023–10–31

Greetings Llama Enthusiasts 🦙!

Another week has zoomed past, and here we are with our latest roundup of updates, features, tutorials, and so much more. Have a noteworthy project, article, or video to share? We’d love to feature it! Reach out to us at news@llamaindex.ai.

Want these updates straight to your inbox? Simply subscribe to our newsletter on our homepage.

🤩 First, the highlights:

1. Revamped Documentation: Overhauled docs for smoother LLM/RAG app development.
2. Contribution Board: Our new board welcomes community-driven LlamaIndex enhancements.
3. Zephyr-7b-beta Insights: Tested and verified for unmatched ReAct agent task efficiency on LlamaIndex.
4. Image Captioning Boost For RAG: LLaVa’s outputs are now supercharged with knowledge-based augmentation. Notebook, Tweet

✨ Feature Releases and Enhancements:

• We introduced Retrieval-Augmented Image Captioning, enhancing LLaVa multi-modal model outputs with knowledge base insights. Notebook, Tweet.
• We introduced the ability to view and set prompts for LlamaIndex modules in just two lines of code. Docs, Tweet.
• We introduced the integration of our OpenAILike class, allowing users to tap into various open-source LLM projects with OpenAI-compatible APIs, irrespective of the model provider. Tweet.
• We introduced Prompt Compression for RAG: with LongLLMLingua, which helps to cut token usage and latency by up to 20x. Notebook, Tweet.
• We introduced a method to refine open-source LLMs like llama2 for structured data outputs. Using LlamaIndex, transform llama2–7b to produce Pydantic objects without PyTorch. Our guide covers synthetic dataset creation, fine-tuning, and RAG pipeline integration. Notebook, Tweet.

🎥 Demos:

• Harshad Suryawanshi did a demo on equity research report generator using LlamaIndex and Streamlit.
• Bharat Ramanathan built Wandbot, a live RAG app enabling chat over Weights & Biases documentation, integrated with Discord and SlackHQ. Key features include periodic data ingestion, custom document and code parsing, model fallback, and logging with Weights and biases.

🗺️ Guides:

• We introduced a revamped documentation structure tailored to guide users from prototyping to production of LLM/RAG apps using LlamaIndex. Dive into our 200+ guides to enhance your app. Docs, Tweet.
• We unveil our new Request For Contribution Github board here. It’s your guide to contribute to LlamaIndex, streamlining community suggestions.
• We released the guide on using the Jina 8k open-source text embedding model with LlamaIndex.
• We introduce our comprehensive survey of llama2-chat models across varying capacities in LlamaIndex. The major insight: While reasoning is enhanced with more parameters, structured outputs remain a challenge. Tweet.
• We share a guide to test the newly released HuggingFace Zephyr-7b-beta model on LlamaIndex RAG/agent tasks, it stood out as the only 7B LLM capable of handling ReAct agent tasks over data.
• We share a new guide on Advanced Prompt Engineering for RAG. Learn about understanding, customizing, and extending RAG prompts, from QA templates to few-shot examples and context/query transformations. Tweet.

✍️ Tutorials:

• Kiran made a blog post on Mastering PDFs: Extracting Sections, Headings, Paragraphs, and Tables with Cutting-Edge Parser.
• Wenqi Glantz gave us an excellent blog post on Optimizing Text Embeddings with HuggingFace’s text-embeddings-inference Server and LlamaIndex.
• Ravi Theja’s blog post delves into NVIDIA Research on RAG vs Long Context LLMs, questioning the necessity of RAG in the presence of long-context LLMs.
• Sudarshan Koirala has a tutorial on Extracting Tables + Texts from .htm pages for RAG Using LlamaIndex.
• Wenqi Glantz also made a second blog post on Multimodal Retrieval with Text Embedding and CLIP Image Embedding for Backyard Birds.

⚙️ Integrations & Collaborations:

• We introduced our new cookbooks in partnership with Gradient AI, enabling effortless fine-tuning of open-source LLMs like Llama 2 and integration into your LlamaIndex RAG pipeline. Docs, Tweet.
• We introduced integration with HuggingFace Inference API which gives access to over 150,000 models. Now you can plugin any conversational, text_generation, feature_extraction endpoints into your LlamaIndex app. Docs, Tweet.

🎥 Webinars:

• Mayo Oshin and Jerry Liu gave a webinar on Unlocking ChatGPT for Business.

📚Workshops:

• Jerry Liu and Simon conducted a Multipart LlamaIndex workshop in collaboration with Anyscale.
• Ravi Theja conducted a day-long workshop on Retrieval Augmented Generation with LlamaIndex.