Content Lead

Build durable chat memory for RAG using ScyllaDB and langchain. By Attila Tóth July 14, 2026 How to replace LangChain's in-memory chat history with ScyllaDB - so your RAG chatbot retains context across restarts and scales across replicas This post demonstrates how to integrate ScyllaDB Vector Search into your LangChain project for RAG use cases, as well as how to use ScyllaDB as a durable conversation memory within LangChain. Background. Large language models are trained on a fixed snapshot of the world. RAG patches that gap by retrieving relevant documents at query time and injecting them into the prompt. The pipeline has two phases: * Index: load documents, split them into chunks, embed each chunk, store embeddings in a vector store. * Retrieve & Generate: enrich and embed the user's question, find the nearest vectors (ANN search), pass the matching chunks as context to the LLM. But RAG alone is not enough. You still need to keep track of all inputs provided by the user. Persistent chat memory. LLMs are stateless. Every call starts with a blank slate unless you replay the conversation history. LangChain's BaseChatMessageHistory abstraction lets you plug in any backend as the storage layer for that history. The default in-memory implementation vanishes on process exit. A database-backed implementation survives restarts, scales across replicas, and lets you inspect or audit conversations later. That's where ScyllaDB comes in. ScyllaDB + LangChain. ScyllaDB is a NoSQL database optimized for high-throughput, low-latency workloads. Combined with LangChain, you can build reliable and always-on AI applications: * High availability: data is automatically replicated across nodes, so there is no single point of failure. The cluster continues serving reads and writes even if a node goes down. * Predictable P99 latency: ANN queries return results fast enough that retrieval doesn't dominate your chain's total latency. * Horizontal scalability: add nodes to the cluster to increase throughput without schema changes or downtime. * Built-in vector search: a native vector<float, N> type and HNSW index are created automatically in ScyllaDB. In this example, embeddings are generated locally with sentence-transformers (all-MiniLM-L6-v2, 384 dimensions). For production use, you can swap in OpenAI embeddings, Cohere, or any other provider supported by LangChain. In each conversation turn, the chatbot is reading data from ScyllaDB and then writing back into it using LangChain. Setup example. With the release of ScyllaDB 2026.2, you can now integrate LangChain with ScyllaDB seamlessly by reusing the existing Cassandra connector. Install dependencies. pip install langchain langchain-community \ sentence-transformers langchain-groq \ langchain-text-splitters cassio \ scylla-driver python-dotenv Environment variables. Copy demo/.env.example to demo/.env and fill in your credentials: # ScyllaDB Cloud SCYLLADB_CONTACT_POINTS=node-0.your-cluster.cloud.scylladb.com SCYLLADB_DATACENTER=AWS_US_EAST_1 SCYLLADB_USERNAME=scylla SCYLLADB_PASSWORD=your-password-here SCYLLADB_KEYSPACE=demo # Groq (LLM) GROQ_API_KEY=gsk_... Retrieve the contact points, datacenter name, username, and password from the Connect tab of your cluster in ScyllaDB Cloud. Connect to ScyllaDB Cloud. from cassandra.cluster import Cluster from cassandra.auth import PlainTextAuthProvider from cassandra.policies import DCAwareRoundRobinPolicy import cassio cluster = Cluster( contact_points=["node-0.your-cluster.cloud.scylladb.com"], auth_provider=PlainTextAuthProvider( "scylla", "your-password"), load_balancing_policy=DCAwareRoundRobinPolicy( local_dc="AWS_US_EAST_1"),) session = cluster.connect cassio.init(session=session, keyspace="demo") The cassio.init call registers the session and keyspace globally so all downstream integrations (the vector store and the chat history) pick it up without further configuration. Although cassio was originally built as a Cassandra integration, it is session-agnostic. It works with whatever driver session you hand it, so passing a session created by scylla-driver works just as well. Integrating ScyllaDB with LangChain vector store. As an example, consider a chatbot that ingests articles, answers questions using retrieved context, and preserves conversation history in the database. from langchain_community.document_loaders import WebBaseLoader from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_community.embeddings import HuggingFaceEmbeddings from langchain_community.vectorstores import Cassandra loader = WebBaseLoader([ "https://docs.scylladb.com/stable/get-started/scylladb-basics.html", "https://docs.scylladb.com/stable/get-started/data-modeling/query-design.html",]) docs = loader.load chunks = RecursiveCharacterTextSplitter( chunk_size=500, chunk_overlap=50, ).split_documents(docs) embeddings = HuggingFaceEmbeddings( model_name="all-MiniLM-L6-v2") vectorstore = Cassandra( embedding=embeddings, table_name="rag_docs") vectorstore.add_documents(chunks) retriever = vectorstore.as_retriever( search_kwargs={"k": 4}) WebBaseLoader fetches and parses each URL into a Document. RecursiveCharacterTextSplitter then breaks each document into 500-token chunks with a 50-token overlap so sentences are not severed at boundaries. Cassandra(table_name="rag_docs") creates the table on first use, including a vector<float, 384> column (matching all-MiniLM-L6-v2's output dimensions) and an HNSW index. Subsequent runs reuse the existing table; you only pay the embedding cost once unless you call add_documents again. Persistent chat memory implementation. from langchain_community.chat_message_histories import CassandraChatMessageHistory def get_chat_history(session_id: str) -> CassandraChatMessageHistory: return CassandraChatMessageHistory( session_id=session_id, table_name="chat_history",) CassandraChatMessageHistory stores each message as a row keyed on session_id. Because rows are written to ScyllaDB, the history survives process crashes, server restarts, and horizontal scaling. Any replica that opens the same session_id sees the same history. Change session_id to start a fresh conversation. Keep it the same to resume where you left off - that is the entire persistence mechanism. Putting it together. from langchain_groq import ChatGroq from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder from langchain_core.output_parsers import StrOutputParser from langchain_core.runnables import RunnablePassthrough from langchain_core.runnables.history import RunnableWithMessageHistory llm = ChatGroq(model="llama-3.3-70b-versatile") ablePassthrough from langchain_core.runnables.history import RunnableWithMessageHistory llm = ChatGroq(model="llama-3.3-70b-versatile") prompt = ChatPromptTemplate.from_messages([ ("system", "You are a helpful assistant. Answer the user's question using the " "retrieved context below.\n\nContext:\n{context}"), MessagesPlaceholder(variable_name="chat_history"), ("human", "{question}"),]) chain = (RunnablePassthrough.assign(context=lambda x: "\n\n".join( d.page_content for d in retriever.invoke(x["question"]))) | prompt | llm | StrOutputParser chain_with_history = RunnableWithMessageHistory( chain, get_chat_history, input_messages_key="question", history_messages_key="chat_history",) config = {"configurable": {"session_id": "user-abc-session-1"}} answer1 = chain_with_history.invoke( {"question": "What is the difference between a partition key and a clustering key in ScyllaDB?"}, config=config,) print(answer1) answer2 = chain_with_history.invoke( {"question": "How does using both keys affect the sort order of the data within a partition?"}, config=config,) print(answer2) # References the prior turn via memory RunnableWithMessageHistory wraps the chain and automatically loads prior turns from get_chat_history before each call, then appends the new turn after. Both storage operations hit ScyllaDB. The second question drills deeper into clustering columns introduced in the first answer. Without persistent memory, the LLM would have no context for what was already explained; with it, the conversation flows naturally across turns. ScyllaDB schema. rag_docs, the vector store: CREATE TABLE demo.rag_docs ( row_id text PRIMARY KEY, attributes_blob text, body_blob text, metadata_s map<text, text>, vector vector<float, 384>,); CREATE CUSTOM INDEX idx_vector_rag_docs ON demo.rag_docs (vector) USING 'vector_index'; CREATE INDEX eidx_metadata_s_rag_docs ON demo.rag_docs (ENTRIES(metadata_s)); chat_history, the message store: CREATE TABLE demo.chat_history (partition_id text, message_id timeuuid, body_blob text, PRIMARY KEY (partition_id, message_id)) WITH CLUSTERING ORDER BY (message_id DESC); Next steps. The full demo code is available on GitHub. You can run it once to ingest and embed the articles, then run it again with the same SESSION_ID to verify that prior conversation turns are loaded from the database. Resources: Post on the ScyllaDB Forum if you have questions!

LangChain and NVIDIA launch NemoClaw Deep Agents Blueprint for enterprise agents. Jul 08, 2026, 11:00 ET The NemoClaw for LangChain Deep Agents blueprint gives enterprises a reference architecture for building open agent systems with benchmark-leading performance and more than 10x lower inference costs SAN FRANCISCO, July 8, 2026 /PRNewswire/ - LangChain today announced the NemoClaw for LangChain Deep Agents blueprint, developed with NVIDIA to help enterprises build, evaluate, and deploy advanced open agent systems. As enterprises move agents into production, the systems they build around the model become valuable IP. Agent memory, workflows, traces, model weights, and tuning data are proprietary intelligence specific to the business. Teams need a way to own that work, improve it over time, and run agents with the performance, cost control, and governance their organizations require. This new blueprint combines LangChain Deep Agents Code, NVIDIA Nemotron 3 Ultra, and NVIDIA OpenShell runtime so teams can customize agents for their workloads, deploy them securely, and run them at lower cost. Benchmark-leading performance with 10x lower cost LangChain's evaluation benchmarks show that enterprises can now get top-performing agents from an open agent stack. In LangChain's agent eval suite, NVIDIA Nemotron 3 Ultra evaluated with LangChain Deep Agents achieved an aggregate score of 0.86 at a cost of $4.48. The next closest performing model cost $43.48, making Nemotron 3 Ultra roughly 10x lower inference cost on this benchmark. The results reflect harness customizations made for Nemotron 3 Ultra. LangChain tuned how the agent uses tools, manages context, and evaluates intermediate steps with Deep Agents. For enterprises, the key takeaway is that agent performance improves when teams tune the model and harness together around the tool-use patterns, context requirements, and workflows specific to their business. "The way to build better agents is to keep improving the system around the model," said Harrison Chase, Co-founder and CEO of LangChain. "Memory, tool use, evaluation, and model behavior compound when teams can tune them together. Our work with NVIDIA shows that enterprises can get strong performance from an open stack while keeping control over the agent systems they're building." Lower inference costs also make it practical to run and evaluate more specialized agents in production. Teams can create agents for specific domains, use evals and traces to measure performance, and adapt the harness as their workflows change. "Super agents have arrived," said Jensen Huang, Founder and CEO of NVIDIA. "With an open model like NVIDIA Nemotron, a LangChain harness, the NVIDIA OpenShell runtime, and a company's own data, every enterprise can build custom agents that understand its business, use its tools, and turn knowledge into action. The future of AI won't be one-size-fits-all - companies will use AI cloud services and build their own AI, shaped by their proprietary data, know-how, and workflows, and run it safely and securely wherever they operate." Harrison Chase and Jensen Huang discuss the blueprint, open agent systems, and the path to lower-cost enterprise AI agents in a fireside chat released today. How the blueprint works The NemoClaw for LangChain Deep Agents blueprint brings together three components essential for building agents for the enterprise: * NVIDIA Nemotron 3 Ultra provides the open-weight model layer for teams that want to customize model behavior for their domains while improving agent performance and lowering cost. * LangChain Deep Agents provides the harness layer for long-running agents, including planning, tool use, memory, and task execution. The Blueprint includes a Deep Agents harness profile is tuned for Nemotron 3 Ultra. * NVIDIA OpenShell provides the runtime layer for secure, governed deployment, helping teams control how agents interact with tools, systems, and data. Together, these components give teams a tuned agent system that can be deployed, measured, governed, and improved in production. Ecosystem support The announcement is supported by partners across the AI infrastructure and enterprise ecosystem, including EY, who is building an implementation practice around the software stack, and Baseten, Fireworks, Nebius, Crusoe, DeepInfra, and Together AI. These partners help enterprises serve Nemotron models in production and adapt the blueprint for business critical applications. "EY clients in regulated industries are ready to move agentic AI out of isolated pilots and into production and are often constrained by governance, security, and the ability to prove control to a regulator or a board. Open agent architectures matter because they give enterprises transparency into how agents operate, control over where data and inference run, and the freedom to deploy on their own terms without committing to a closed stack. By delivering the NVIDIA NemoClaw blueprint, which incorporates together with LangChain technology, EY teams help give clients a secure, sandboxed foundation for always-on agents that can meet enterprise standards for auditability and risk from the first deployment." - Geoff Vickrey, Global Chief Commercial Officer, NVIDIA, EY "Production agents need inference that is fast, reliable, and cost-efficient at scale. We have optimized NVIDIA Nemotron models on Baseten to deliver high throughput and low latency on NVIDIA hardware, so teams get strong price-performance without operating the infrastructure themselves. Delivering Nemotron through the NemoClaw blueprint with LangChain gives enterprises a clear path to run open agentic models in production with the performance and economics these workloads demand." - Philip Kiely, Head of Developer Relations, Baseten. "Agentic workloads make many model calls per task, so inference speed and cost directly determine whether an agent is viable in production. Fireworks serves NVIDIA Nemotron models with the throughput and price-performance that high-volume agent systems require, tuned for the tool calling and reasoning patterns these workloads depend on. Offering Nemotron through the NemoClaw blueprint with LangChain gives enterprises an efficient, open foundation they can scale with confidence." - Lin Qiao, CEO and Cofounder, Fireworks AI. "The next challenge for enterprise AI is running complex agentic workloads economically at production scale. Nebius was built for that challenge. Our AI-native cloud gives customers dedicated infrastructure optimized for high-performance inference and cost-efficient scaling. By offering NVIDIA Nemotron models through the NemoClaw blueprint with LangChain, we're making it easier for organizations to deploy and scale open agentic AI across their business." - Roman Chernin, Chief Business Officer, Nebius Availability The NemoClaw for LangChain Deep Agents Blueprint is available now. Enterprises can access the blueprint to evaluate the stack for their own workloads. About LangChain LangChain powers the full agent development lifecycle - building, testing, deploying, and monitoring - so AI teams can improve their agents systematically. LangSmith Engine accelerates this cycle, automatically surfacing and fixing issues to improve agents over time. LangSmith is neutral by design, so teams can customize their own stack to optimize on cost and performance as the landscape evolves. More than 7,000 customers, including NVIDIA, Bridgewater, LinkedIn, Workday, Harvey, and Rippling trust LangSmith to build and manage their agents. SOURCE LangChain

LangChain releases openwiki: an open-source AI agent and CLI that writes and maintains your repo documentation. OpenWiki is a new open-source CLI from LangChain that automatically generates and maintains codebase documentation designed specifically for AI coding agents. AIDeveloper44 Team OpenWiki automates codebase documentation to give AI agents structured, retrievable context. * LangChain's OpenWiki is an open-source CLI that generates structured codebase wikis for AI coding agents. * It automatically updates files like AGENTS.md or CLAUDE.md to point agents to this documentation, replacing bloated context files with a retrieval model. * A built-in GitHub Action automates daily documentation updates via pull requests based on recent Git commits. * The tool supports multiple inference providers, custom gateway URLs, and optional LangSmith tracing. Introduction to OpenWiki. LangChain has released OpenWiki, an open-source command-line interface (CLI) tool designed to write and maintain documentation for codebases. Rather than generating standard documentation for human developers, OpenWiki targets AI coding agents. The tool constructs a structured knowledge base that coding agents can navigate, addressing the common problem of outdated or bloated agent instruction files. The problem with single-file instructions. In the current ecosystem of AI-assisted development, tools like Claude Code, Cursor, and OpenAI Codex rely on root-level markdown files - most notably AGENTS.md or CLAUDE.md - to understand project context. These files act as a persistent memory layer, providing the agent with coding standards, architectural guidelines, and project requirements. However, developers frequently overload these instruction files. Storing hundreds of pages of project history and architectural context in a single file severely inflates the agent's context window. When a large language model (LLM) processes a massive context file during every chat session, its ability to locate and adhere to specific, granular instructions diminishes. This approach increases API costs and degrades the agent's performance, as models struggle to consistently execute tasks when burdened with excessive, irrelevant context. The structured retrieval approach. OpenWiki implements an alternative architecture. Instead of centralizing all information into one instruction file, it creates a dedicated openwiki/ directory within the repository. This directory functions as a localized wiki, populated with summaries, structural maps, and cross-references organized specifically for an LLM's parsing capabilities. To connect this wiki to the coding agent, OpenWiki automatically updates the existing AGENTS.md or CLAUDE.md files. If these files do not yet exist, the CLI creates them. It then appends specific prompting instructions that direct the agent to search and reference the openwiki/ directory when it requires context. This shifts the agent's workflow from loading the entire repository context at startup to retrieving necessary documentation on demand. Source Codebase OpenWiki CLI LLM Processing openwiki/ Dir Structured Docs Context Pointers Agent Retrieves Diagram: OpenWiki analyzes the codebase to generate structured documentation and updates instruction files, allowing agents to retrieve context on demand. Static documentation becomes outdated quickly, which creates friction when agents rely on it. OpenWiki addresses this by offering continuous maintenance through a GitHub Action workflow. Developers can copy the provided openwiki-update.yml workflow file into their repository's .github/workflows/ directory. Configured to run on a daily schedule, this action executes the CLI in update mode (openwiki -update). The tool analyzes recent Git commits and file differences, identifies what parts of the codebase have changed, and rewrites the relevant wiki files. Following the update, the action automatically opens a pull request with the new documentation, ensuring a human can review the automated changes before they are merged. CLI usage and tracing. The CLI is installed globally via npm using the command npm install -g openwiki. Running openwiki -init begins an interactive setup process where the user configures their preferred model and API key. If an openwiki/ directory already exists, the initialization command refreshes the existing documentation based on repository changes rather than rebuilding it entirely from scratch. The tool offers multiple operational modes: * Interactive Mode: Running openwiki keeps the interface open for continuous prompt refinement. * Single Execution: The -p or -print flag allows for a one-shot, non-interactive run that prints the assistant's output and exits. * Update Mode: The -update flag refreshes documentation based on recent codebase changes. Additionally, OpenWiki supports optional integration with LangSmith. By providing a LangSmith API key during setup, developers can trace the underlying agent's execution paths while it generates documentation. This data is logged to a LangSmith project named "openwiki," allowing for performance auditing and debugging. Provider flexibility and gateway support. OpenWiki is designed to be provider-agnostic. Out of the box, it supports OpenRouter, Fireworks, Baseten, OpenAI, Anthropic, and generic OpenAI-compatible providers. The system includes pre-defined configurations for several current LLMs (such as GLM 5.2, Kimi K2.6, and Sonnet 5). Users can also specify custom model IDs based on their own requirements. For organizations utilizing self-hosted models, proxies, or API gateways (such as a LiteLLM gateway), OpenWiki allows custom endpoint routing. Users can define alternative base URLs, such as ANTHROPIC_BASE_URL or OPENAI_COMPATIBLE_BASE_URL, alongside their API keys. All configuration settings and secrets are saved locally on the user's machine in a ~/.openwiki/.env file. References & Sources