My AI Projects Were Quietly Disagreeing With Each Other

A few weeks ago I was building a report and noticed two numbers that should have been identical. Same metric, same work, two different figures. One came from one Claude Project. The other came from a second Project I'd used earlier in the week. Neither was wrong, exactly. They were just from different points in time, and I was the only thing standing between them and a report that would go to my skip level manager (uh, that would be bad).

I caught it this time. That was the part that worried me. My quality control was a guy remembering to squint and read each line carefully.

I've written before about building things that fail before they work. This is a different problem. This is the one AI handed me after the building was done.

How I Created My Own Mess

Over the past year I've gone deep on Claude Projects. If you don't use them, picture a Project as a dedicated workspace with its own instructions and its own knowledge base. I have a lot of them. One for reporting, one for strategy work, one for content, and on down the list. Keeping them siloed is mostly a feature. The strategy Project doesn't need to know about the content Project, and the separation keeps each one on point.

Most of the time.

The trouble is that a few of them overlap. A handful of my key numbers live in more than one Project, because more than one workflow touches them. So when I update knowledge sources, which I do frequently, I update the number in one place and the old version keeps living in the others.

Here's the pattern that was hurting my workflows. I took a process, automated part of it, and AI has helped speed it up. Then a few months later I go deeper and it streamlines even more. Great! Now that single metric of time saved exists in three versions across my Projects, and only the newest one is true. Nothing about that is dishonest. Numbers change. Vendors change their terms. That's corporate life. But every stale copy is a small landmine waiting for the day I pull from the wrong Project under a deadline for a report to leadership.

And the only detector in the system was me. On every ad hoc report, part of my brain was running a background process: is this number current, and which Project has the real one? That's cognitive load I didn't have room for.

The Solutions That Didn't Fit

I'm stubborn about problems like this, so I tried several things before I landed anywhere.

First I tried a GitHub connection, with the idea that a single repo could feed my Projects. Workable in theory. It didn't fit how I actually work day to day.

Then I tried what I started calling "digital 1:1s." Once a week I'd sit down and sync my Projects by hand, walking through my main ones and updating whatever had drifted. It worked but then I would let a few Projects slip because how many 1:1s can I have with robots. Any fix that depends on me being perfectly disciplined every single week is not a fix. It's a chore I will eventually skip.

Then I asked IT if I could connect Claude to SharePoint and OneDrive, so the numbers could pull from one governed place. They said no. Fair enough. Governance exists for reasons, and "let the AI read everything in our document store" is not a small ask. But it closed the cleanest door.

None of these were bad ideas. For someone else, in a different setup, any one of them might be the right answer. They just didn't fit my constraints. That distinction turned out to be the whole point, but I didn't see it yet.

The Reframe

I was still trying to make my Projects agree with each other. That was the mistake.

Sitting with it, I asked a different question. What if they don't have to agree? What if I stop trying to keep a dozen knowledge bases in sync, and instead build one thing whose only job is to notice when they don't?

I didn't need a system that fixes. I needed a system that catches. A second pair of eyes that never gets tired and never forgets which number is current.

That reframe is the entire reason this worked. Cal Newport writes about this, I think it's in "Slow Productivity" but might be "Digital Minimalism," either way, both are solid reads.

What I Built

I built a Claude Skill that holds the canonical version of my key numbers in one file. I call it the source of truth.

It doesn't write my reports. That's not its job. Its job is to watch. When I'm building something that references my key metrics, the Skill fires, checks what's in my draft against what it knows to be current, and flags anything that doesn't match. Then it points me toward where the stale number is probably coming from, so I can go fix it at the root.

I'll be honest about the trade, because the honest part is what makes it useful. The Skill cannot reach into my Projects and correct them. The old numbers still live in those knowledge bases until I go update them myself. So the maintenance didn't disappear. It moved. Instead of trying to remember every Project that might be stale, I keep one file current on purpose, and I fix the others reactively when the Skill points at them.

That's a trade I'll take every time. Updating one file on a calendar reminder is a thing I can actually do (I set a reminder for the last Friday of each month). Remembering all of them was a thing I was already failing at.

There's something I find quietly funny about the result. I used AI to check AI. The fragmentation came from leaning hard into these tools, and the fix was another one of these tools pointed back at the first. The judgment about which number is true still has to come from me. The Skill just makes sure I don't forget to apply it.

The Part That Actually Matters

The Skill is not the lesson. If you walk away with "build a source of truth Skill," you grabbed the wrong end.

Here's the real one. AI is pushing all of us into a world where the right solution is custom.

The tools are getting commoditized. Everybody has the same models now. What is not commoditized is the fit between a problem and its solution, and that fit is specific to you. A GitHub repo might be exactly right for one person. A direct SharePoint connection might be perfect for another. For my problem, my constraints, and the way I work, a Claude Skill won. There was no universal answer to look up. I had to find mine.

And finding mine took the least futuristic thing imaginable. I tried something, it didn't fit. I tried the next thing, it didn't fit either. I got told no. I kept going until the reframe showed up. That is not an AI skill. That is grit, and it predates every tool I used.

I keep coming back to the 10-80-10 idea for exactly this reason. AI takes the middle 80%, and it keeps getting better at it. The first 10%, defining the real problem, and the last 10%, refusing to quit before you have actually solved it, are still mine. The better the models get at the middle, the more those two ends are worth.

AI will hand you new problems. It will not hand you the patience to solve them. That part is still yours.

Pick the problem that is quietly costing you. Try the obvious fix. When it does not fit, do not assume the answer is not out there. Assume you have not found yours yet.