Every audit I have planned started the same way. A room, a process owner, and documentation describing a process nobody runs that way anymore. The narrative was written in 2023 for the last engagement. The control matrix has been copied forward through three cycles. The person who understood the reconciliation left in March.

So the first two weeks of a six week audit go into rebuilding knowledge the organization already had and lost.

On July 15, Cerebras published how they solved a version of this for their engineers. Chip company, not audit function, but read past the infrastructure and the problem is ours. Their information had scattered across Slack, wikis, code, incidents and team databases, and people kept asking the same three questions. Where is X. Who knows about Y. What is Z. What they built now fields more than 15,000 questions a day.

What struck me was not the scale. It was that their design decisions map, almost item for item, onto how audit works.

They gave up on the single source of truth

Cerebras did not consolidate. They note that someone regularly proposes recording everything in one platform, and that “the dream of a single source of truth” rarely survives contact with reality. Knowledge gets created wherever it is convenient. Slack for the argument, Jira for the status, the code for the actual behaviour.

Every audit function I know has tried the other way at least once. The repository that will hold all process documentation, all control descriptions, all prior findings. It decays within a year because nobody generates knowledge there. The real reason the tolerance is 5% sits in an email thread from 2024.

Cerebras built one connector per source and one shared table instead, so everything stays queryable through a single interface. Translate that to audit and the source list writes itself. Prior workpapers, the control matrix, policies, the risk register, Jira and ServiceNow tickets, incident records, minutes, ERP configuration and system logs. None of it moves. It gets read where it lives.

Distillation is evidence synthesis

The part that made me sit up is what they do to a conversation before storing it. They do not index the raw transcript. A model reads the whole thread and extracts a structured artifact: the question someone would actually search for, a summary, the resolution, and the systems involved. That is what gets stored, and they report accuracy improved once they stopped keeping the raw text.

Any auditor who has read a 47 page process narrative knows what that is. It is what we do by hand, badly, under time pressure. You read the narrative and extract four things. Here is the control. Here is who owns it. Here is what can go wrong. Here is the evidence that would show whether it worked. The rest is context you never open again. Doing that once, at ingestion, across every process document in the organization is different in kind from doing it per engagement.

Scoping is an engagement

Searching everything stopped being useful as their corpus grew. Compiler engineers did not want infrastructure runbooks in their results, so Cerebras introduced projects: named bundles of sources, with a default scope per user.

That is an engagement. A procurement audit needs the policy, the P2P control matrix, three years of prior findings, the vendor master change log and part of the risk assessment. It does not need the HR onboarding runbook. The same source can sit in several scopes, which is how a shared incident channel already behaves.

Who actually runs this process

Another of their tools is called who_knows. It surfaces people with demonstrated expertise on a topic, inferred from what they have answered rather than from a title.

Every experienced auditor has learned to distrust the org chart. The person listed as control owner signs the attestation. The person who knows what happens when the interface fails at month end is two levels down and has been there eleven years. Finding them is one of the least documented skills we have.

The retrieval design is how auditors already think

This is the parallel I find most useful, and the easiest to skip because it looks technical.

Cerebras runs several retrieval methods at once and fuses the results, because each covers a weakness in the others. Full-text search catches the exact tokens semantic similarity blurs, the error strings and flag names. Vector search catches paraphrase, connecting a question to an answer written in different vocabulary. Inverse document frequency lets rare terms outrank filler, so “sounds good, thanks” stops competing with a real explanation. Age decay means that when two answers are otherwise equal the newer one wins, because the older thread may describe infrastructure that no longer exists.

Read that list again as evidence rules. Some audit evidence is exact and has to match precisely: a control ID, a policy clause, a journal reference. Some is semantic, as in does this process description match what we observed. Some is rare and therefore high signal, because one odd exception in a clean population tells you more than a hundred conforming items. And all of it decays.

We already triangulate this way. We do it in our heads, inconsistently, and we lose the reasoning when the senior rolls off the job.

The planning phase, permanently half finished

The IIA’s Global Internal Audit Standards ask us to understand the organization’s governance, risk management and control processes before we plan an engagement — the internal audit equivalent of what ISA 315 asks of a financial statement auditor. We treat that as an activity with a start date. It is not. It is a standing asset we rebuild from scratch every engagement and then discard, because it lives in the heads of the people who did the fieldwork and it leaves when they rotate.

A knowledge base that continuously ingests organizational data means the understanding never fully resets. When you scope the engagement, the planning phase is already partly done. Not the judgment. The retrieval.

Which is where this meets the audit orchestrator I have been building. The orchestrator directs the work. It takes a process, surfaces risks, drafts tests, runs them, hands you a workpaper. But a model without organizational context produces generic findings. It will tell you segregation of duties in procurement matters. Thank you. Connect that same model to your control matrix, your prior findings, your incident history and your policy text, and the output changes character. It stops describing controls in general and starts describing yours.

The orchestrator is the direction. The knowledge base is the intelligence. I had been treating those as one project and they are not. The knowledge layer is the foundation, and worth building first, because it earns its keep even if you never automate a single test.

One warning. An audit knowledge base holds draft findings and conclusions not yet released, and a system that ingests everything inherits every access problem in the organization. Cerebras built authorization and auditing into theirs from the start. Build this without that layer and your first finding is about yourself, which is the argument I made about functions that cannot list their own agents.

Where I would start

  • List one recurring audit’s real sources. Not the ones named in the methodology. What you actually used last cycle. That is your first scope.
  • Distill before you store. Extract control, owner, failure mode and evidence from each narrative. Keep that, not the PDF.
  • Ask every process owner who else answers questions about this, and record the answer.
  • Timestamp everything and let it decay. Evidence with no date is evidence you cannot weight.
  • Settle access before ingestion, not after.

If a new auditor joined on Monday and asked how revenue recognition actually works here, what would you hand them? If the answer is a slide deck from 2023 and a phone number, the knowledge is not in your function. It is in three people’s heads, and you are one resignation away from planning your next audit from scratch.

If you are building something like this inside an audit function, I would like to hear how you are scoping it. Get in touch.

Sources and further reading: