Weaviate Podcast

Siddharth Gollapudi, a researcher at UC Berkeley, joins the Weaviate Podcast to discuss in-context retrieval and his paper "Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale." The idea is simple and radical: instead of encoding documents into embeddings and building a nearest neighbor index, put the entire corpus into an LLM's context window and use attention itself as the retriever. The conversation opens by separating this from generative retrieval, which memorizes documents into model parameters and needs gradient updates every time the corpus changes, and from listwise re-ranking, which Siddharth frames as an easier...

Alexis, Manya, and Niloofar from humans& join the Weaviate Podcast to introduce Persimmon, a user model built to simulate how humans actually behave in multi-turn, multi-party conversations. Persimmon is explicitly not an assistant, a companion, or a Character AI-style stand-in, it is a research preview aimed at faithfully capturing the distribution of human behavior. This includes the natural friction of frustration, excitement, and group dynamics that assistant chatbots trained to be helpful never exhibit. Alexis, Manya, and Niloofar bring a striking mix of backgrounds to the problem: AI tutoring and student modeling, programming languages for high-performance computing, and privacy...

Alex Ledbetter is the founder of Delegance Brokerage, an AI-native commercial insurance brokerage he built entirely on his own. The conversation opens with his origin story: an internship on a political risk, credit, and bond underwriting team in New York, where he watched a three-trillion-dollar portfolio run off the back of Excel and realized commercial insurance brokers earning 15–35% annual commissions could be disintermediated the way Robinhood disintermediated retail stock brokers. After raising $200K and securing 190 state licenses in 45 days, he spent seven months teaching himself to build with AI coding tools. Alex compares this process to hammering sheet metal, wa...

Sam O'Nuallain joins the Weaviate Podcast to discuss AutoIndex, research from UMass Amherst and Databricks on learning representation programs for retrieval. Instead of tuning the retriever or re-ranker, AutoIndex asks how the data itself should be represented: a two-agent system, an analysis agent and a code agent, writes and refines Python programs that chunk, enrich, and reorganize a corpus to optimize downstream metrics like Recall and nDCG.The conversation opens with why indexing is such a natural target for code optimization. Frontier LLMs are exceptional at writing code, a representation program applies cheaply across an entire corpus without passing...

Alex Zhang, a PhD student at MIT, joins the Weaviate Podcast to discuss Recursive Language Models (RLMs), a new abstraction for designing agent harnesses. Instead of the standard ReAct-style loop that stuffs every tool observation into an ever-growing prompt, an RLM treats the prompt as a variable in a program. The model writes code that manipulates its own context and spawns recursive LLM calls over pieces of it. The published headline was long-context performance, but the deeper intention, inspired by how DSPy programmers decompose tasks, is letting the model do that decomposition itself, relieving context pressure so each call...

Mathew Jacob, lead author of "Drowning in Documents: Consequences of Scaling Reranker Inference" and now a PhD student in ML systems at the University of Washington, joins the Weaviate Podcast to unpack one of the most surprising results in modern search: cross-encoder rerankers get worse as you give them more documents. The paper began during his Databricks internship, where scaling reranking past roughly 100 documents sent recall@10 plummeting, a result so counterintuitive he assumed it was a bug.The conversation digs into why this happens, reframing rerankers through the lens of boosting, rather than being strictly stronger than first-stage retrievers...

Weaviate co-founders Bob van Luijt and Etienne Dilocker return to the Weaviate Podcast to celebrate seven years of building the company, answering questions submitted by the community. The conversation opens with what excites them most in AI right now: Etienne on agentic coding and the "Moore's law" of how long models can sustain autonomous loops, and Bob on world models, new architectures that could slash training energy costs, open source frontier models, and inference on new chips.From there, the discussion dives into taste and the "AI slopification" problem, why AI-generated emails, websites, and decks all look the same...

Dr. Bradley Allen brings five decades of AI history into a deep conversation on knowledge engineering, neurosymbolic AI, and the future of enterprise intelligence. The discussion begins with the boom-and-bust cycle of rule-based expert systems, AI winters, and why today’s large language model wave may be different. The conversation then turns to how knowledge is organized in practice, from personal piles of papers searched on demand to formal knowledge graphs built with classes, relations, ontologies, A boxes, T boxes, description logic, and subsumption-based reasoning. Allen explains why semantic web and biomedical ontology successes still leave unresolved questions about co...

Başak Eskili joins the Weaviate Podcast to explore how one of the world’s largest travel platforms adopted vector search, retrieval-augmented generation, and agentic AI at production scale. The conversation begins with Booking.com’s shift from keyword matching to semantic retrieval as internal teams needed embeddings, similarity search, and eventually GenAI RAG workflows. Başak explains why OpenSearch was a practical first step on AWS, how adoption grew across teams, and why hundreds of millions of embeddings, strict latency requirements, complex filtering, and rising concurrency pushed the platform toward Weaviate.The discussion then moves into Booking.com’s partne...

Dr. Nandan Thakur returns to the Weaviate Podcast fresh off defending his dissertation to discuss the evolution from neural retrieval to agentic search and his new work on Orbit, a synthetic training data pipeline for search agents. The conversation opens with reflections on his PhD journey, tracing the field's shift from ColBERT-style models and sparse retrievers through RAG and into today's agentic search paradigm where LLMs iteratively search, reason, and refine.The discussion dives deep into how Orbit generates multi-hop, riddle-style training queries using DeepSeek's API on a personal laptop over four to six months, making high-quality search agent...

Zijian Chen and Xueguang Ma from the University of Waterloo join the Weaviate Podcast to discuss AgentIR and why retrieval systems need to be redesigned from the ground up for AI agents. The conversation opens with a striking reframe: agents have become the primary consumers of search, inserting themselves as middleware between humans and information. Humans used to query search engines directly, now they delegate to ChatGPT, which searches on their behalf. This means retrieval algorithms are no longer optimized for their actual users.The discussion distinguishes reasoning-intensive retrieval from reasoning-aware retrieval. Reasoning-intensive tasks like BRIGHT involve single-hop queries...

Shreya Shankar from UC Berkeley joins the Weaviate Podcast to discuss data agents, the Data Agent Benchmark, and DocETL. The conversation opens with defining what a data agent actually is, not just text-to-SQL over a single table, but an AI system that can reason across dozens of heterogeneous databases, flat files, and knowledge repositories to answer complex organizational questions. Shreya explains why this multi-database reality makes existing benchmarks insufficient, motivating the Data Agent Benchmark where the best-performing agent achieves only 34–37% pass@1 accuracy.

From there, the discussion dives into where agents fail. They don't explore data properly, th...

Amélie Chatelain and Antoine Chaffin from LightOn are leading the way in the next generation of search powered by Multi-Vector representations and Late Interaction. The podcast begins with what motivates them to work on Multi-Vector Search, continuing to discuss particular details such as the combination between lexical and semantic search, as well as bi-encoder speed with cross encoder accuracy. This discussion continues to present insights about training multi-vector models and how they differ from their single-vector predecessors. The conversation continues into particular successes of Late Interaction such as code, reasoning-intensive, and multimodal retrieval. Agents are great at searching w...

Doug Turnbull and Trey Grainger join the Weaviate Podcast to discuss all things AI-Powered Search! The conversation kicks off with designing search experiences, not all search queries are the same! Sometimes the user knows exactly what they want (a product ID, a specific file), other times they're exploring a broad category, and other times they need to compare and contrast options. AI is now making it possible to dynamically construct UIs around search results, moving toward what Trey describes as a "Minority Report"-style future where visualizations adapt on the fly to the query and the data.From there...

Thomas van Dongen is the head of AI engineering at Springer Nature and the creator of Pyversity! Pyversity is a fast, lightweight open-source Python library for diversifying retrieval results. Retrieval systems often return highly similar items. Pyversity efficiently re-ranks these results to encourage diversity, surfacing items that remain relevant but less redundant. It implements several popular diversification strategies such as MMR, MSD, DPP, and Cover with a clear, unified API.

Matthew Russo is a Ph.D. student at MIT where he is researching the intersection of AI and Database Systems. AI is transforming Database Systems. Perhaps the biggest impact so far has been natural language to query language translations, or Text-to-SQL. However, another massive innovation is brewing. AI presents new Semantic Operators for our query languages. For example, we are all familiar with the WHERE filter. Now we have AI_WHERE, in which an LLM or another AI model computes the filter value without needing it to be already available in the database!

`SELECT * FROM podcasts AI...

Xiaoqiang Lin is a Ph.D. student at the National University of Singapore. During his time at Meta, Xiaoqiang lead the research behind REFRAG: Rethinking RAG-based Decoding. Traditional RAG systems use vectors to retrieve relevant context with semantic search, but then throw away the vectors when passing the context to the LLM. REFRAG instead feeds the LLM these pre-compute vectors, achieving massive gains in long context processing and LLM inference speed! REFRAG makes Time-To-First-Token (TTFT) 31x faster and Time-To-Iterative-Token (TTIT) 3x faster, boosting overall LLM throughput by 7x while also being able to handle much longer contexts!

<...

This episode dives into Weaviate's partnership with SAS! We are super excited about our recent collaboration on the SAS Retrieval Agent Manager (RAM), featuring a first party integration with Weaviate! The podcast dives into all sorts of aspects of Enterprise AI adoption from what has changed, to what has NOT changed with recent breakthroughs in AI systems!

Charles Pierse is the Director of the Weaviate Labs team, where he has recently lead the GA release of the Weaviate Query Agent. The podcast begins with the journey from alpha to GA release, discussing unexpected lessons and the collaborations between teams at Weaviate. Continuing on the product design, we cover the design of the Python and TypeScript clients and how to think about response models with Agent products. Then diving into the tech, we cover several different aspects of the Query Agent from question answering with citations, to schema introspection and typing for database querying, multi-collection routing, and...

Lakshya A. Agrawal is a Ph.D. student at U.C. Berkeley! Lakshya has lead the research behind GEPA, one of the newest innovations in DSPy and the use of Large Language Models as Optimizers! GEPA makes three key innovations on how exactly we use LLMs to propose prompts for LLMs, (1) Pareto-Optimal Candidate Selection, (2) Reflective Prompt Mutation, and (3) System-Aware Merging. The podcast discusses all of these details further, as well as topics such as Test-Time Training and the LangProBe benchmarks used in the paper! I hope you find the podcast useful!

Maarten Grootendorst is a psychologist turned AI engineer who has created BERTopic and authored "Hands-On Large Language Models" with Jay Alammar. The rise of LLMs and Agents are transforming many areas of software! This podcast dives deep into their impact on Topic Modeling! Maarten designed BERTopic from the start with modularity in mind -- letting you ablate embedding models, dimensionality reduction, clustering algorithms, and more. This early insight to prioritize modularity makes BERTopic incredibly well structured to become more "Agentic". An "Agentic" Topic Modeling algorithm can use LLMs to generate topics or topic descriptions, as well as contrast them...

Hailey Joren is a Ph.D. student at UCSD! Hailey and collaborators at Duke University and Google have recently published Sufficient Context: A New Lens on Retrieval Augmented Generation Systems in ICLR 2025! There are so many interesting nuggets to this work! Firstly, it really helped me understand the difference between *relevant* search results and sufficient context for answering the question. Armed with this lens of looking at retrieved context, Hailey and collaborators make all sorts of interesting observations about the current state of Hallucination. RAG unfortunately makes the models far less likely to hallucinate, and the existing RAG benchmarks...

Nandan Thakur is a Ph.D. student at the University of Waterloo! Nandan has worked on many of the most impactful works in Retrieval-Augmented Generation (RAG) and Information Retrieval. His work ranges from benchmarks such as BEIR, MIRACLE, TREC, and FreshStack, to improving the training of embedding models and re-rankings, and more!

Multi-vector retrieval offers richer, more nuanced search, but often comes with a significant cost in storage and computational overhead. How can we harness the power of multi-vector representations without breaking the bank? Rajesh Jayaram, the first author of the groundbreaking MUVERA algorithm from Google, and Roberto Esposito from Weaviate, who spearheaded its implementation, reveal how MUVERA tackles this critical challenge.

Dive deep into MUVERA, a novel compression technique specifically designed for multi-vector retrieval. Rajesh and Roberto explain how it leverages contextualized token embeddings and innovative fixed dimensional encodings to dramatically reduce storage requirements while maintaining high retrieval...

AI agents are getting more complex and harder to debug. How do you know what's happening when your agent makes 20+ function calls? What if you have a Multi-Agent System orchestrating several Agents? Anand Kannappan, co-founder of Patronus AI, reveals how their groundbreaking tool Percival transforms agent debugging and evaluation. Percival can instantly analyze complex agent traces, it pinpoints failures across 60 different modes, and it automatically suggests prompt fixes to improve performance. Anand unpacks several of these common failure modes. This includes the critical challenges of "context explosion" where agents process millions of tokens. He also explains domain adaptation for...

How do you ensure your AI systems actually do what you expect them to do? Leonard Tang takes us deep into the revolutionary world of AI evaluation with concrete techniques you can apply today. Learn how Haize Labs is transforming AI testing through "scaling judge-time compute" - stacking weaker models to effectively evaluate stronger ones. Leonard unpacks the game-changing Verdict library that outperforms frontier models by 10-20% while dramatically reducing costs. Discover practical insights on creating contrastive evaluation sets that extract maximum signal from human feedback, implementing debate-based judging systems, and building custom reward models that align with enterprise...

Ben walks us through Box's three-layer infrastructure puzzle: First, the mind-boggling base infrastructure (think millions of interactions per second and trillions of files). Second, their unique multi-tenant security challenge - unlike most SaaS platforms, Box users share content across company boundaries, making traditional tenant isolation impossible. And third, ensuring AI respects all these complex permissions while still delivering value. The podcast then dives further into how vector embeddings can balloon file sizes - a few hundred bytes of text can require 4-6KB of vector data storage! We also dig into why RAG remains essential despite growing context windows...

Hey everyone! Thanks so much for watching another episode of the Weaviate Podcast! Dive into the fascinating world of structured outputs with Will Kurt and Cameron Pfeiffer, the brilliant minds behind Outlines, the revolutionary open-source library from .txt.ai that's changing how we interact with LLMs. In this episode, we explore how constrained decoding enables predictable, reliable outputs from language models—unlocking everything from perfect JSON generation to guided reasoning processes.Will and Cameron share their journey to founding .txt.ai, explain the technical magic behind Outlines (hint: it involves finite state machines!), and debunk misconceptions around structured generation pe...

Synthetic Data: The Building Bocks of AI's Future! Hey everyone! I am SUPER EXCITED to publish the 118th episode of the Weaviate Podcast featuring David Berenstein and Ben Burtenshaw from HuggingFace! This podcast explores the intricacies of synthetic data generation, detailing methodologies such as data augmentation, distillation, and instruction refinement. The conversation delves into persona-driven synthetic data, highlighting applications like Persona Hub, and discusses algorithms to enhance diversity, complexity, and quality of generated data. Additionally, they cover integration with Hugging Face’s ecosystem, including Argilla for annotation, AutoTrain for fine-tuning, and advanced data exploration tools like the Data Studio an...

Hey everyone! Thank you so much for watching the 117th episode of the Weaviate podcast! In this episode, we dive deep into the cutting edge of AI agent development with Sarah Wooders, co-founder and CTO of Letta AI. Emerging from Berkeley's Sky Computing Lab, Sarah and her team have pioneered a revolutionary approach to stateful agents - AI systems that genuinely remember both you and themselves across extended conversations. The conversation explores how the groundbreaking MemGPT project evolved into Letta's comprehensive Agent Development Environment (ADE), which empowers developers to build truly persistent AI experiences. Sarah shares powerful insights on...

Hey everyone! Thank you so much for watching another episode of the Weaviate Podcast! I am SUPER excited to welcome Matt Biilmann, Co-Founder and CEO of Netlify, as well as Sebastian Witalec and Charles Pierse from Weaviate to discuss Agent Experience! You have probably heard about how you can connect LLMs to external software tools. This supercharges the capabilities of AI systems and what they can do. So what does that mean for you as a software developer?This podcast explores different ideas around designing software user experiences for Agents as well as Humans. How do we write documentation...

Hey everyone! Thank you so much for watching the 115th episode of the Weaviate Podcast featuring Shirley Wu from Stanford University!

We explore the innovative Avatar Optimizer—a novel framework that leverages contrastive reasoning to refine LLM agent prompts for optimal tool usage. Shirley explains how this self-improving system evolves through iterative feedback by contrasting positive and negative examples, enabling agents to handle complex tasks more effectively.

We also dive into the STaRK Benchmark, a comprehensive testbed designed to evaluate retrieval systems on semi-structured knowledge bases. The discussion highlights the challenges of unifying textual and re...

Hey everyone! Thank you so much for watching the 114th episode of the Weaviate Podcast featuring Amanpreet Singh, Co-Founder and CTO of Contextual AI! Contextual AI is at the forefront of production-grade RAG agents! I learned so much from this conversation! We began by discussing the vision of RAG 2.0, jointly optimizing generative and retrieval models! This then lead us to discuss Agentic RAG and how the RAG 2.0 roadmap is evolving with emerging perspectives on tool use. Amanpreet continues to further motivate the importance of continual learning of the model and the prompt / few-shot examples -- discussing the limits of...

Hey everyone! Thank you so much for watching the 113th episode of the Weaviate Podcast with Karan Goel from Cartesia AI! Cartesia AI is leading the AI world in text-to-speech models! As exciting as these new applications in speech generation are, Cartesia is also building around an incredibly exciting new neural network architecture that cuts across all of AI -- State Space Models. State Space Models (SSMs) present a new approach to modeling long sequences circumventing the quadratic attention bottlenecks of transformers. In the podcast, we discuss Karan's perspectives around end-to-end modeling, long context and Multimodal processing, building and...

Hey everyone! Thank you so much for watching the 112th episode of the Weaviate Podcast! This is another super exciting one, diving into the release of the Vertex AI RAG Engine, its integration with Weaviate and thoughts on the future of connecting AI systems with knowledge sources! The podcast begins by reflecting on Bob's experience speaking at Google in 2016 on Knowledge Graphs! This transitions into discussing the evolution of knowledge representation perspectives and things like the semantic web, ontologies, search indexes, and data warehouses. This then leads to discussing how much knowledge is encoded in the prompts themselves and...

Hey everyone! I am SUPER EXCITED to publish the 111th Weaviate Podcast with Aravind Kesiraju from Morningstar! Aravind is a Principal Software Engineer who has lead the development behind the Morningstar Intelligence Engine! There are so many interesting aspects to this, and if you are building Agentic systems that would benefit from a high-quality financial retrieval API, you should check this out right now! The podcast dives into all sorts of ingredients that went into building this system: from custom RAG data pipelines with content management system integrations and embedding task queues, to exploring new chunking strategies, tool marketplaces...

Hey everyone! Thank you so much for watching the 110th episode of the Weaviate Podcast! Today we are diving into Snowflake’s Arctic Embedding model series and their newly released Arctic Embed 2.0 open-source model, additionally supporting multilingual text embeddings. The podcast covers the origin of Arctic Embed, Pre-training embedding models, Matryoshka Representation Learning (MRL), Fine-tuning embedding models, Synthetic Query Generation, Hard Negative Mining, and Single-Vector Embeddings Models in the cohort of Multi-Vector ColBERT, SPLADE, and Re-rankers.

Hey everyone! Thank you so much for watching the 109th episode of the Weaviate Podcast with Erika Cardenas! Erika, in collaboration with Leonie Monigatti, have recently published "What is Agentic RAG". This blog post that was even covered in VentureBeat with additional quotes from Weaviate Co-Founder and CEO Bob van Luijt! This podcast continues the discussion on all things Agentic RAG, covering the basics of Agents, how Agentic RAG changes the game compared to Vanilla RAG systems, Multi-Agent Systems and CrewAI / OpenAI Swarm, Letta, DSPy, and many more! The podcast also anchors by discussing Agentic Generative Feedback Loops and...

JSON mode has been one of the biggest enablers for working with Large Language Models! JSON mode is even expanding into Multimodal Foundation models! But how exactly is JSON mode achieved?

There are generally 3 paths to JSON mode: (1) constrained generation (such as Outlines), (2) begging the model for a JSON response in the prompt, and (3) A two stage process of generate-then-format.

I am BEYOND EXCITED to publish the 108th Weaviate Podcast with Zhi Rui Tam, the lead author of Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language...

Hey everyone! Thank you so much for watching the 107th episode of the Weaviate Podcast! This one dives into SWE-bench, SWE-agent, and most recently SWE-bench Multimodal with John Yang from Stanford University and Carlos E. Jimenez from Princeton University! One of the most impactful applications of AI we have seen so far is in programming and software engineering! John, Carlos, and team are at the cutting-edge of developing and benchmarking these systems! I learned so much from the conversation and I really hope you find it interesting and useful as well!