Communication is painful for every organization, and there's even a numerical threshold where it immediately starts to break down. Whether you call it a knowledge base, a company brain, or a wiki, you should build a knowledge loop to kill the pain. http://x.com/i/article/2082982852086661120
An internal knowledge base holds the IP, secrets, and tribal knowledge that makes up a company’s DNA. So why do knowledge bases usually suck?
@cerebras published an excellent post on how they built their internal knowledge base. It garnered 2.5M views in its first week, which is more than double the views of @AnthropicAI's post on the history of Claude Code a week earlier.
Despite the power of AI at our disposal, the average knowledge base is more of a trash fire than ever. Yesterday’s knowledge tools are not built for today’s workflows.
I led knowledge management at Stripe and Uber through their hypergrowth phases. I’ve seen how a great knowledge base leads to faster onboarding, greater productivity, higher retention.
In this post, I’ll explain:
I. Why most knowledge bases suck
The number one problem with knowledge bases is that there’s no ownership. Without accountability, everything decays.
Secondly, they’re hidden. You can reverse-engineer Stripe’s world-class documentation or study Vercel’s beautiful frontend design because they’re public. You can’t tour another company’s internal knowledge base.
There are three common failure modes within a knowledge base:
-
There’s no information on the topic.
-
There’s missing or outdated information.
-
There are conflicting pieces of information.
Your knowledge base might suck if:
-
information is scattered and out of date
-
your agents aren’t getting the right context
-
many of your questions are still answered by teammates
Like any engineering problem, you should measure the state of your knowledge base to see where the issues are and how you’re improving over time.
Every hard problem needs to start with a quick dopamine hit to hook people into solving it. The best engineers are typically masochists, so they get a dopamine hit from seeing the problems through data.
This is why Falconer built the first knowledge observability tool, Knowledge Health, to solve this exact problem.
II. The anatomy of a great knowledge base
Start with the codebase
Cerebras picked a focused set of data sources for their knowledge base: code, wiki, incidents, Slack.
This is counter to today’s popular kitchen-sink approach of connecting everything and calling it a company brain. The company brain approach is alluring because you get access to more information, but the quality of that information drops drastically when it conflicts with itself.
The best approach is a strong knowledge graph, architected around your most reliable, authoritative source: the codebase. Code is structured, tested, and versioned. LLMs love this context and knowledge graphs feast on it. The proof is in the benchmarks.
From there, move up the ladder of sources. Documentation is often unmaintained and not organized or tested the way code is.
Slack is typically the most popular source, but also the noisiest and most ephemeral. Cerebras rightfully takes into account the surrounding conversation to gather context. But there are two important things Cerebras doesn’t say about Slack.
First, knowledge typically gets trapped in DMs and scattered across channels which agents can’t access. Second, Cerebras stores distilled knowledge chunks in a database, but that isn’t a surface humans can browse, edit, and maintain. Important information from Slack should make its way to documents, since it’s the best collaboration surface for humans and agents.
Re-think search
Great knowledge is useless if no one can find it. Google has billions of queries per day to learn from. But traditional search techniques don’t work within organizations, because the data sets are small. A wildly popular company page or search query might get a couple hundred hits in a week. You cannot optimize search just from usage the way Google can, because the signal isn’t there.
But engineers do constantly ask questions that rhyme, so query popularity within an org follows a power law. You can see the bulk of search topics based on company priorities: release plans, goals, roadmaps, OKRs, incidents. If you track popularity, you can bolster results and answer quality for 80-90% of searches.
There’s a simple, yet powerful principle that leads to great search: when you curate the underlying knowledge, you get much better results. Delete old cruft, consolidate conflicting sources, and update the docs.
Use documents as a reliable source
Cerebras doesn’t spend much time talking about their docs, which according to their graphic is a Confluence wiki feeding the pipeline. However, documents are king! Documents are where ideas are collaborated on, systems are architected, history is gathered, and where the company’s future is defined. Backlinks in docs tell the story of interconnectedness, and are increasingly where agents publish their artifacts and memories from coding sessions. The opportunity is to treat documents as first-class data, with attribution, version control, citations, and accuracy.
Documentation should be an authoritative source, rather than a knowledge landfill.
Information architecture
Information architecture shapes the browsing experience for users and agents. If your information is organized, it’s easier to find and contribute to. It’s like the table of contents for your company.
Knowledge graph over projects
Cerebras scopes results by project, which is a good start, but projects within growing companies are too fluid to depend on as a scope. Projects are constantly being launched, changed, and deprecated. A good knowledge graph supersedes projects dynamically by creating clusters of related teams, documents, and tasks.
Automate updates
Knowledge without automated maintenance is doomed to fail. Imagine your codebase without tests, reviews, a filesystem, version control, and monitoring. If something is worth writing down, it’s worth keeping up to date. But it’s impossible to do this without the right tools. When Stripe hit 10,000 people, we had 500,000 documents. It takes sophisticated tools to manage all this knowledge.
Scanning your documents for outdated information and automatically making updates when code changes is the only approach that scales.
Invest in the authoring experience
A good authoring experience is the key to getting engineers to capture knowledge. Some engineers like writing docs as code, which is also great for agents. Some engineers like to write in a UI, which is ideal for simplicity and collaboration.
This is why supporting both use cases is the only approach that works in practice. But you need a two-way sync so you can edit documents in the UI with changes synced back to a repo. This way your agent can go nuts documenting in the repo, and they get synced back to a human-readable UI.
III. Rolling it out
https://pmc.ncbi.nlm.nih.gov/articles/PMC8103230/#:~:text='Dunbar's%20number'%20is%20the%20notion,other%20individuals%20in%20the%20group
Cerebras employees ask 15,000 questions a day, which is about 15 questions per person. What’s interesting is the list of questions they chose to illustrate the problem. These are questions that we all struggled with long before having access to AI:
-
Where do I find X?
-
Who owns Y?
-
What is Z?
You can also add these to the list:
-
What did we decide?
-
Who made the decision?
-
Why did we decide this?
These questions are getting even harder to answer because codebases are growing larger and there’s less understanding of the code being generated. No one wants to slow down and build less, but what we can do is improve our understanding of our systems and organizations.
There’s an organizational number when, like clockwork, the pain of a poor knowledge base starts to become unbearable: Dunbar’s number. You can hold about 150 stable relationships in your head, and right around 150 people is when an org stops knowing who’s working on what. I’ve watched it happen over and over without fail.
https://x.com/cerebras/status/2077822555159945507?s=20
Create accountability
Unlike most parts of a company, knowledge base owners are non-existent, disempowered, or lacking the right tools.
Give the overall knowledge base a DRI, and ensure individual teams own their sections.
This is why we believe in the concept of “document owner.” This person sees and approves auto-updates to their documents, can see the issues across documents, and when they change teams or leave the company, their documents get assigned to new owners.
Use a phased approach
Cerebras allows teams to build connectors to the knowledge base. That means the new knowledge base could launch without waiting for every team to migrate. But those teams carry their own maintenance burden, which becomes motivation to migrate to the core system later.
For most companies, I recommend starting even smaller. Start with one mission-critical team and overhaul their docs. Timebox it to a two-week sprint. With the right approach, the resulting docs will feel like night and day in comparison to other teams’ docs. Then everyone will follow suit with a golden example to follow.
The Holy Grail is within reach
Investing in a knowledge base creates a self-reinforcing flywheel. Customers get features and fixes faster. Managers get more productivity out of their teams. ICs are blocked less often.
Cerebras references the Holy Grail of a “single source of truth,” but says it rarely works in practice.
Things change constantly. People won’t contribute through a surface that doesn’t fit how they work. Engineers want to leave feedback in PR comments, PMs want to comment in a Google Doc, managers dish out feedback in Slack. If you make contribution painful, people will stop contributing.
This is why you need a system that can support all the pieces for a living knowledge system:
-
Generate the docs
-
Organize the docs
-
Monitor the docs
-
Update the docs
The important tradeoff to make is providing the central system responsible for collecting, connecting, and maintaining all the knowledge. Falconer offers the Holy Grail out of the box, where there’s one place for humans and agents to contribute and consume. Everything we believe and discuss here is built into the Falconer platform to allow every team to leverage their most important asset: their knowledge.
