How Clawdbot Remembers Everything
- Source: https://x.com/manthanguptaa/status/2015780646770323543?s=46
- Mirror: https://x.com/manthanguptaa/status/2015780646770323543?s=46
- Published: 2026-01-26T13:35:32+00:00
- Saved: 2026-01-28
Content
Clawdbot is an open-source personal AI assistant (MIT licensed) created by Peter Steinberger that has quickly gained traction with over 32,600 stars on GitHub at the time of writing this blog. Unlike ChatGPT or Claude, which run in the cloud, Clawdbot runs locally on your machine and integrates with chat platforms you already use, like Discord, WhatsApp, Telegram, and more.
What sets Clawdbot apart is its ability to handle real-world tasks autonomously: managing emails, scheduling calendar events, handling flight check-ins, and running background jobs on a schedule. But what caught my attention was its persistent memory system, which maintains 24/7 context retention, remembering conversations and building upon previous interactions indefinitely.
If you’ve read my previous posts on ChatGPT memory and Claude memory, you know I am fascinated by how different AI products approach memory. Clawdbot takes a fundamentally different approach: instead of cloud-based, company controlled memory, it keeps everything local, giving users full ownership of their context and skills.
Let’s dive into how it works.
How Context is Built
Before diving into memory, let’s understand what the model sees on each request:
The system prompt defines the agent’s capabilities and available tools. What’s relevant for memory is Project Context, which includes user-editable Markdown files injected into every request:
These files live in the agent’s workspace alongside memory files, making the entire agent configuration transparent and editable.
Context vs Memory
Understanding the distinction between context and memory is fundamental to understanding Clawdbot.
Context is everything the model sees for a single request:
Context is:
Ephemeral - exists only for this request
Bounded - limited by the model’s context window (e.g., 200K tokens)
Expensive - every token counts toward API costs and speed
Memory is what’s stored on disk:
Memory is:
Persistent - survives restarts, days, months
Unbounded - can grow indefinitely
Cheap - no API cost to store
Searchable - indexed for semantic retrieval
The Memory Tools
The agent accesses memory through two specialized tools:
- memory_search
Purpose: Find relevant memories across all files
Returns
- memory_get
Purpose: Read specific content after finding it
Returns:
Writing to Memory
There is no dedicated memory_write tool. The agent writes to memory using the standard write and edit tools which it uses for any file. Since memory is just Markdown, you can manually edit these files too (they will be re-indexed automatically).
The decision of where to write is prompt-driven via AGENTS.md:
Automatic writes also occur during pre-compaction flush and session end (covered in later sections).
Memory Storage
Clawdbot’s memory system is built on the principle that “Memory is plain Markdown in the agent workspace.”
Two-Layer Memory System
Memory lives in the agent’s workspace (default: ~/clawd/):
Layer 1: Daily Logs (memory/YYYY-MM-DD.md)
These are append-only daily notes that the agent writes here throughout the day. The agent writes this when the agent wants to remember something or when explicitly told to remember something.
Layer 2: Long-term Memory (MEMORY.md)
This is curated, persistent knowledge. Agent writes to this when significant events, thoughts, decisions, opinions, and lessons are learned.
How the Agent Knows to Read Memory
The AGENTS.md file (which is automatically loaded) contains instructions:
How Memory Gets Indexed
When you save a memory file, here’s what happens behind the scenes:
sqlite-vec is a SQLite extension that enables vector similarity search directly in SQLite, no external vector database required.
FTS5 is SQLite’s built-in full-text search engine that powers the BM25 keyword matching. Together, they allow Clawdbot to run hybrid search (semantic + keyword) from a single lightweight database file.
How Memory is Searched
When you search memory, Clawdbot runs two search strategies in parallel. Vector search (semantic) finds content that means the same thing and BM25 search (keyword) finds content with exact tokens.
The results are combined with weighted scoring:
Why 70/30? Semantic similarity is the primary signal for memory recall, but BM25 keyword matching catches exact terms that vectors might miss (names, IDs, dates). Results below a minScore threshold (default 0.35) are filtered out. All these values are configurable.
This ensures you get good results whether you are searching for concepts (“that database thing”) or specifics (“POSTGRES_URL”).
Multi-Agent Memory
Clawdbot supports multiple agents, each with complete memory isolation:
The Markdown files (source of truth) live in each workspace, while the SQLite indexes (derived data) live in the state directory. Each agent gets its own workspace and index. The memory manager is keyed by agentId + workspaceDir, so no cross-agent memory search happens automatically.
Can agents read each other’s memories? Not by default. Each agent only sees its own workspace. However, the workspace is a soft sandbox (default working directory), not a hard boundary. An agent could theoretically access another workspace using absolute paths unless you enable strict sandboxing.
This isolation is useful for separating contexts. A “personal” agent for WhatsApp and a “work” agent for Slack, each with distinct memories and personalities.
Compaction
Every AI model has a context window limit. Claude has 200K tokens, GPT-5.1 has 1M. Long conversations eventually hit this wall.
When that happens, Clawdbot uses compaction: summarizing older conversation into a compact entry while keeping recent messages intact.
Automatic vs Manual Compaction
Automatic: Triggers when approaching context limit
You will see: 🧹 Auto-compaction complete in verbose mode
The original request will retry with compacted context
Manual: Use /compact command
/compact Focus on decisions and open questions
Unlike some optimizations, compaction persists to disk. The summary is written to the session’s JSONL transcript file, so future sessions start with the compacted history.
The Memory Flush
LLM-based compaction is a lossy process. Important information may be summarized away and potentially lost. To counter that, Clawdbot uses the pre-compaction memory flush.
The memory flush is configurable in clawdbot.yaml file or clawdbot.json file.
Pruning
Tool results can be huge. A single exec command might output 50,000 characters of logs. Pruning trims these old outputs without rewriting history. It is a lossy process and the old outputs are not recoverable.
JSONL file on disk: UNCHANGED (full outputs still there)
Cache-TTL Pruning
Anthropic caches prompt prefixes for up to 5 minutes to reduce latency and cost on repeated calls. When the same prompt prefix is sent within the TTL window, cached tokens cost ~90% less. After the TTL expires, the next request must re-cache the entire prompt.
The problem: if a session goes idle past the TTL, the next request loses the cache and must re-cache the full conversation history at full “cache write” pricing.
Cache-TTL pruning solves this by detecting when the cache has expired and trimming old tool results before the next request. Smaller prompt to re-cache means lower cost:
Session Lifecycle
Sessions don’t last forever. They reset based on configurable rules, creating natural boundaries for memory. The default behaviour is reset everyday. But there are other modes available.
Session Memory Hook
When you run /new to start a fresh session, the session memory hook can automatically save context:
Conclusion
Clawdbot’s memory system succeeds because it embraces several key principles:
- Transparency Over Black Boxes
Memory is plain Markdown. You can read it, edit it, and version control it. No opaque databases or proprietary formats.
- Search Over Injection
Rather than stuffing context with everything, the agent searches for what’s relevant. This keeps context focused and costs down.
- Persistence Over Session
Important information survives in files on disk, not just in conversation history. Compaction can’t destroy what’s already saved.
- Hybrid Over Pure
Vector search alone misses exact matches. Keyword search alone misses semantics. Hybrid gives you both.
References
Clawdbot Documentation - Official docs covering setup, configuration, and all features
GitHub Repository - Source code, issues, and community contributions
If you found this interesting, I would love to hear your thoughts. Share it on Twitter, LinkedIn, or reach out at guptaamanthan01[at]gmail[dot]com.
You can find more blogs from me on https://manthanguptaa.in/
Link: http://x.com/i/article/2015775451810246656