Skip the AI strategy and start with your own knowledge
Last November I was existentially anxious about AI. The tools were changing every month. People I respect were using it in ways I didn’t understand. I run a data science company that helps mission-driven organizations make decisions with data, and AI was beginning to change our work. But I didn’t have a vision for the future, and that was scary.
A little pit of fear can explode into panic, so I embraced the anxiety and used research to smother it like a grenade. I wanted to understand exactly what AI could and should do for our company, our team, and our clients. I just wanted to do something useful, without the hype and without the FOMO.
There are a lot of first steps an organization can take. Most of them you’ve probably heard about: write an acceptable use policy. Stand up a data governance process. Invest in training. We already had each of those at January Advisors. They tell your team what tools they can use and how to use them well.
But we had a more fundamental question that wasn’t answered by any of these first steps: what can AI do for our work today, on our own data, with what we already know?
That question is what sent me looking for infrastructure. Along the way, I built a January Advisors knowledge base. That’s what this post is about.
Two truths
Technology often runs top down. Vision deck. Roadmap. Vendor selection. Pilot program. Organizational adoption. MIT found that 95% of enterprise GenAI pilots deliver no measurable impact. When you skip the deck and start building, you bump into two truths that help explain why.
Truth #1: You can’t imagine the use cases before seeing AI at work. It’s difficult to articulate a vision before you can see what’s possible. The interesting use cases come from understanding your operation at the process level, and watching AI run against real workflows. They don’t even sound like AI use cases until you’re in them. “Sort my email” didn’t sound like one at first, but now it’s indispensable.
Truth #2: AI won’t sort out what’s useful from what’s not. This applies in two ways. The first is that AI is just guessing at what’s important, but you know the truth. A grant proposal sent in on a whim does not have the same weight as one sent to your primary funder. The second is that you need a data governance process to classify what AI can touch. That requires people, and AI isn’t a person.
These aren’t reasons to wait. They’re reasons to start small and harbor great ambition.
Start with a knowledge base
A knowledge base is what you give an AI to ground its answers in what your organization actually knows. Without one, an agent is guessing from the open internet. But you can give it specific context that improves its results dramatically.
For our knowledge base, I started with our previous proposals and project debriefs. I found files that weren’t restricted or private, and I put them in a new folder. Then I provided access, instructions, and a whole lot of questions.
This was a wrestling match. I locked horns with Claude until I could pin it down for real, useful answers. It took some time and calibration, but when it was ready, years of project history became one picture. What used to be guesswork could suddenly be verified.
The knowledge base is useful on its own. We use it to onboard new employees, store original research, keep track of new ideas, and answer questions about our project history.
But it also acts as a power pack for other AI agents. Now we can:
- Forecast staffing and capacity in minutes
- Chat about financial projections and scenarios using real-time information from several systems
- Organize new research and see the threads between ideas
- Manage my hyperactive email down to inbox zero
- Scan news about our clients every day
A knowledge base is the foundation. The fun stuff is what follows.
Five steps to do it
We’ve been running a version of this with three clients now. The process keeps evolving. But here’s how it goes today.
1. Define what should go in. Interview colleagues at the same time.
Two tracks happen in parallel.
With the client lead, we decide what should be in the knowledge base. We bring examples from our own practice: post-mortem lessons learned, decision logs, methodology briefs. Seeing them helps people recognize what they already have. Most organizations have more of this than they realize, and most of it is trapped in someone’s head, inbox, or Google Drive.
We interview people across the organization. What are your difficult processes? What do you wish you knew in real time? What disappears when someone leaves? These conversations show us how the work actually runs. That’s where the use cases live.
2. Pick the documentation exercises that will do the most work.
The interviews usually highlight a few documents that have the most payoff. A consolidated methodology brief. An annotated decision log. A handoff for a position that no one ever documented.
If they don’t exist yet, we write them down. They become the raw material of the knowledge base. Doing the hard work up front also forces a quiet clarity: much of the value of the exercise is what the client notices about their own thinking.
3. Set up an opt-in shared folder.
Start with an empty folder. Give the agent access to that folder and nothing else. Files enter only after they’ve been reviewed and moved in. This is your new walled garden.
It sounds trivial, but it’s where most of the work gets done. Every concern about AI “seeing things it shouldn’t see” gets resolved at the folder level. Your lawyer will love you. Once the risk is controlled, the rest of the work can move quickly.
4. Co-create a simple set of data governance rules.
Data governance sounds fancy and important, and to those of us in the industry, it is. But it’s also pretty simple. It classifies information so you know what you can do with each file. Restricted? Don’t share. Public? Fair game.
We put together a lightweight governance guide that lets the team decide quickly what goes into the knowledge base. It fits on one page. It names what’s allowed, what isn’t, and who decides when it’s ambiguous.
With a guide like this, data governance for AI stops being a six-month committee deliverable and becomes a one-line check that anyone can run.
5. Build the knowledge base. Interrogate it together.
We build the knowledge base from the approved files. Then we sit with the client and run an explicit testing pass. The testing has three jobs.
Demonstrate accuracy. We probe the knowledge base with real questions the client cares about. We calibrate until the answers are right, with the source documents pulled up next to them.
Find the gaps. Where the knowledge base doesn’t know. Where the answer is thin. Where the documentation we hoped existed actually doesn’t. The gaps go back into step 2 or step 3 as a working punch list.
Review the synthesized views. The knowledge base does two things: it retrieves documents and it synthesizes information across them. The synthesis is where an agent can quietly editorialize. Sometimes it’s accurate, or funny, or technically correct but substantially wrong. This is also the most fun.
By the time the testing pass is done, the knowledge base is real, the governance rule is written, and a use-case list has surfaced from the work itself. None of it required an AI strategy committee.
Pick a topic and start
You can do this too. A knowledge base doesn’t have to be comprehensive. Choose a limited topic, like public grant proposals or donor personas. Then ask it about the patterns.
A knowledge base on day one beats an AI strategy on month nine.
If you want a hand, drop me a line.