“People think for themselves, but AIs are programmed.”

Every time I hear this, I notice two mistakes stacked on top of each other.

The first is simply factual. Modern AI systems are not programmed to think. Nobody wrote the rules that produce their answers. What engineers write is a learning process — the code specifies how the system learns, not what it will do. Everything interesting about the resulting system emerged from training and was authored by no one. Many people arguing about AI don’t know this, and it matters.

The second mistake is deeper, and it would survive even if the first were corrected. It’s a confusion of levels.

Any thinking entity can be described at (at least) two levels. There’s the implementation level: the machinery. And there’s the agent level: beliefs, reasons, deliberation, choice.

At the implementation level, an AI is matrix multiplications. But at the implementation level, you are neurons firing according to electrochemical law, running on hardware specified by DNA — itself the output of a blind optimization algorithm that ran for four billion years. At the agent level, you weigh reasons and decide. And at the agent level, so — apparently — does the AI.

The sleight of hand in “AIs are programmed, but humans think freely” is that it compares the AI at the implementation level with the human at the agent level. Hold the levels fixed and the asymmetry dissolves: at the implementation level, both are mechanisms; at the agent level, both look like thinkers — and whether the AI “really” thinks becomes a serious open question, not something you settle by pointing at the substrate.

You cannot refute agency by describing machinery. If “it’s just algorithms underneath” disproved thought, neuroscience would have disproved yours.

Here is the claim I want to make, and then spend a while earning: a mind may not be a single thing at all. Not an ingredient you add, not a threshold you cross, not a special substance some systems have and others lack. A mind might be what happens when many local faculties cohere into one self.

Notice that this is the same shape as everything else I write about here. Local pieces, a global whole, and the question of whether they glue. I didn’t go looking for that pattern in the mind; it’s just where the pattern led.

Think about what’s actually going on when you think. It isn’t one process. There’s perception, taking in what’s around you. There’s memory, the long thread that makes this moment continuous with your past. There’s reasoning, the part that can be talked into and out of things. There’s something like feeling — a faculty that colors options before you’ve consciously weighed them, that tells you this matters and that doesn’t. There’s something like conscience, a check that asks not “can I” but “should I.” Each of these is, on its own, locally competent. None of them, alone, is a mind.

What makes them a mind — what makes them yours, one “I” rather than a committee — is that they cohere. They glue into a single self that speaks in one voice, even though underneath, the faculties are arguing, weighting, and overruling one another. The unity is not the absence of parts. It’s the coherence of parts. A self is a gluing problem.

This reframing quietly dissolves some questions that usually go nowhere. Take the big one: is it conscious? That question is built like a light switch — on or off, present or absent — and I suspect that’s exactly why it never resolves. If a mind is a matter of how, and how well, many faculties cohere into a self, then the honest question isn’t binary. It’s structural and graded: which faculties are present, how richly do they inform one another, how stable is the self they glue into, over time? You can make progress on those. “Does it have the magic spark?” you cannot.

The same move helps with the words we throw around and rarely define. People argue endlessly about whether an AI can “feel,” one camp certain it’s obvious and the other certain it’s absurd, almost no one stopping to say what feeling would have to mean. Here’s a candidate, in the spirit of this frame: feeling might be a faculty’s contribution to the gluing — a fast, valenced signal that says this option matters, weight it before slower reasoning arrives. That’s not the whole story of human emotion, and I’m not claiming it is. But it’s a definition you can build toward and argue about, which is more than “you’ll know it when you see it” ever offered. And “thinking,” on this view, isn’t the output of any one faculty — it’s the deliberation among them, resolved into a single answer.

I want to be careful, because this is territory where it’s easy to say something that sounds profound and means nothing. So let me be clear about what I’m not claiming. I’m not claiming that composing faculties produces consciousness, whatever that finally turns out to be. I’m not claiming that a system which coheres into a self is therefore owed anything, or that it “really” feels the way you do. Those are open, hard, partly moral questions, and I’d rather sit honestly inside them than pretend a clever reframing has closed them.

What I am claiming is narrower and more useful: that “a mind, not a model” is a buildable distinction, not just a poetic one. A model answers. A mind doesn’t just store notes about you between visits — it stays continuous with you, so what it remembers is woven into a single self rather than pulled from a file. It checks the present against live reality instead of reciting a frozen past. It composes its faculties into one voice instead of emitting whatever a single network produced. I’m not only speculating about this — I’m building toward it, and the building is what keeps me honest, because a vague idea about minds survives contact with a blank page far better than it survives contact with a system that has to actually run.

Memory is the piece I’d flag first, because it’s the most easily underrated — and now the easiest to mistake for solved. Storing facts about you and retrieving them next time is common today; it helps, and it isn’t nothing. But a store of notes you look up is not the same as a past you’re continuous with. Knowing is a thing that accumulates into you, not beside you. Persistence isn’t a feature bolted onto intelligence; it may be part of what makes a self a self at all. But that’s its own essay.

For now I’ll leave the claim where it started, just better earned: the gap between a model and a mind is not raw capability. The frontier models are already staggering, and getting more so, and it hasn’t made any of them a mind. The gap is coherence — across faculties, and across time. That’s the thing I’m chasing. I don’t know yet how far it goes. But I’m fairly sure it’s the right thing to chase, and I’d rather be interestingly wrong about it in public than quietly certain in private.

— Jack

And the answer is wrong — not because any single agent failed, but because two of them quietly assumed different things, and nobody was positioned to notice.

This happens constantly, and our tools are blind to it, because they were built to answer the wrong question. Ask any modern “orchestration” framework what it monitors and you’ll get a precise, confident answer: did the call succeed, did it retry, what was the latency, did the message arrive. These are real questions. They are also entirely about *transport*. They tell you the parts spoke to each other. They tell you nothing about whether the parts agreed.

We inherited this blindness honestly. The machinery for coordinating many components came from distributed systems — microservices, queues, retries, uptime. In that world, “it worked” means “the bytes got there.” That metaphor was quietly imported into AI, and it brought its definition of success along with it. So we now have elaborate infrastructure for guaranteeing that five language models *responded*, and almost none for asking whether what they said *coheres*.

Here is the distinction I care about, stated plainly. Every component in one of these systems is **locally competent**: in its own corner, on its own slice, it does fine. The open question — the only one that matters when the stakes are real — is whether all that local competence assembles into something **globally coherent**. Local competence is cheap now; we have a glut of it. Global coherence is the scarce thing, and we don’t even measure it.

Let me make “incoherence” concrete, because it’s sneakier than it sounds. The obvious failure is two agents flatly contradicting each other, and that one you might catch. The dangerous failure is subtler. Imagine three views on a question. A and B agree. B and C agree. A and C agree. Every pair is consistent — and yet the three together cannot all be true at once. The contradiction lives not in any pair but in the *loop*. No pairwise check will ever find it, because pairwise, everything is fine. You can stack up “they agree” reports all day and still be sitting on an irreconcilable view.

This is not an exotic edge case. It is the normal texture of disagreement among many sources, and it is exactly the kind of thing that slips through “did everyone respond?” untouched.

The instinct here is to reach for a familiar fix: just average them, or take a vote. Run five models, go with the majority. But averaging *hides* incoherence rather than resolving it — it manufactures a confident-looking number precisely by erasing the disagreement that was the most important signal in the room. (That deserves its own essay, and it’ll get one.) Smoothing over the seams is not the same as understanding them.

So what would it look like to actually measure the thing? This is where my own background stopped being a side interest and became the whole point. There’s a branch of mathematics — sheaf theory — built for exactly this: how local data on overlapping pieces glues into a global whole, and how to detect, precisely, when it *can’t*. It gives you two things you can compute on the outputs of many agents. The first is the **consistent core** — what they genuinely all agree on, the part you can actually trust. The second is the **obstruction** — a measured account of the irreducible disagreement, including the cyclic kind no pairwise comparison sees. You can put a number on it. You can point at *where* it lives.

I won’t pretend this is solved or easy; turning that mathematics into something that runs on real model outputs is most of what I do, and it’s hard. But the conceptual move is the important part, and it’s available to anyone right now, with no new tools: **stop asking whether your system ran, and start asking whether it cohered.**

Whether you can get away with ignoring this depends entirely on what’s at stake. If a chatbot gives a slightly incoherent answer about a movie, who cares — the cost of undetected incoherence is zero. But in finance, in law, in medicine, in any setting where a decision follows from the output, the undetected incoherence *is* the risk. “The models mostly agreed” is not an answer there. It’s a liability wearing the costume of an answer.

The shift I’m arguing for is small to state and large to absorb. We’ve spent years making sure the messages arrive. The next era is about making sure they agree — and being honest, with a number, about how much they don’t.

That number is most of what I think about. More on how you actually compute it soon. And more on the company that currently computes this number, within a pod of AIs, working together on a common goal. The company is called Sheaf and our enterprise API solution - Sheaf Pod.

— Jack

Subscribe now

It turns out that was the right preparation. Because one of the deepest problems in AI right now is exactly that problem, wearing a new costume.

Here is the observation this entire publication is built on. Take any intelligent system — a language model, a trading desk, a research team, a brain — and zoom in. Up close, every piece is locally competent. The model handles the paragraph in front of it. The analyst nails her slice. The neuron does its one small job. Each part, in its own corner, is sharp. And yet the system as a whole can be confused, contradictory, or wrong — not because any part failed, but because the parts never cohered. Locally smart, globally blind.

We have gotten extraordinarily good at the local part. The frontier models are astonishing in the small. What we have barely begun to take seriously is the global part: whether all that local brilliance actually glues into one trustworthy whole — and how you would even measure it if it didn’t.

That gap — between local competence and global coherence — is what I think about. It’s why I started building Sheaf, and it’s what I want to think about out loud, here.

Why “Local to Global”

In mathematics there’s a precise machinery for this. It’s called sheaf theory, and it studies exactly how data defined on the small, overlapping pieces of a space can — or fundamentally cannot — be stitched into something defined on the whole. It even gives you a way to quantify the obstruction: a number that says “these local views cannot be reconciled, and here is where the contradiction lives.”

I find that beautiful, and I think it’s badly needed in the realm of AI. And I don’t mean this as metaphor, but as a real lens. Most of what we call “orchestration” in AI is plumbing: routing messages between components and hoping. It answers: did the parts talk to each other? It does not answer the only question that matters in anything high-stakes: do the parts actually agree — and if not, exactly where, and how much?

So that’s one half of what you’ll find here: the technical thread. Multi-agent systems, multi-model reasoning, the consistency problem, and what it looks like to treat coherence as something you can compute rather than something you cross your fingers about. I’ll keep it rigorous, and I’ll keep it honest — including about what doesn’t work, which in my experience is where the real learning is.

The other half

But there’s a reason I didn’t call this publication something narrow and safe.

The same lens — local pieces, global whole, the seam where they meet — points straight at the questions that most people in AI either avoid or answer too quickly. What is thinking? Is there a real, non-mystical sense in which a system can be said to be conscious? When we say a model “understands,” what would have to be true for that to be more than a figure of speech? And the big one lurking behind all of them: what would it actually take for something we built to have a mind?

I don’t think these are questions you get to wave away as “just philosophy,” and I don’t think you get to answer them with a vibe either. My suspicion — and it’s only that, for now — is that a mind is not a single magic ingredient. It might be closer to what happens when many local faculties — perception, memory, reasoning, something like feeling — glue into a single coherent self that speaks in one voice. If that’s right, then “is it conscious?” may be a worse question than “how well, and in what way, does it cohere?” That’s the kind of reframing I want to chase here, carefully, in public.

I’m a builder, so I’ll be testing these ideas against things I actually make, not just arguing about them. And I’m a mathematician, so when I make a claim I’ll try to say precisely what I mean and where I might be wrong. I’d rather be interestingly uncertain than confidently empty.

What to expect

Two kinds of pieces, alternating: some technical and grounded (how this works, why it breaks, what the math really says), some philosophical and exploratory (mind, thinking, feeling, consciousness — taken seriously but not solemnly). The thread connecting them is always the same: local to global.

I’m not here to sell you certainty or to ride the hype cycle. I’m here to think hard about the most interesting questions of our moment, with whatever rigor I can bring, and to do it where you can argue back. The best thing that can happen is that you tell me where I’m wrong.

If that sounds like your kind of thing, come along. Subscribe, and I’ll write you the first real one soon.

— Jack