Dear Inferential Loopers,
Brian writes about transformers and cognitive science. I write notes about the sector I love – University Advancement, the fundraising and alumni engagement function at colleges and universities, where we build lifelong relationships between institutions and the people who care about them. And we’ve somehow arrived at the same conclusion from completely different directions: predicting isn’t understanding, and the human in the loop isn’t optional.
I’ve spent thirty years in this field. Here is what I actually believe from the practitioner’s end.
Predicting isn’t understanding – and I feel that every day.
When I ask an AI tool to draft something for a senior audience with limited sector familiarity, it defaults – repeatedly, predictably – to insider language. Fluent, confident, and wrong for the room. It’s completing the pattern of “Advancement document” rather than understanding the actual problem: translating sector logic for a sceptical VP Finance who thinks in capital investments, not donor pipelines.
Catching that failure requires knowing what the right answer looks like. Which requires thirty years of sector knowledge the model doesn’t have. The autocomplete is impressive. The understanding lives elsewhere.
The kea parrot is my new favourite metaphor for donor relationships.
That kind of open-ended, context-specific, never-before-seen adaptation Brian identifies as genuine intelligence is exactly what experienced Advancement professionals do in every complex donor relationship.
When a long-time donor mentions something in passing – a life situation, a grandchild’s ambitions, a shift in their values – an experienced gift officer recognizes a signal no historical dataset could have predicted. It’s not autocomplete. It’s understanding built from thousands of relationship interactions and genuine care about this specific human being. An algorithm can surface a name. It cannot know what to do with a conversation that has never happened before.
The most valuable design input in building fundraising analytics has not been historical data alone – it’s been structured conversations with experienced fundraisers about what makes them pause. That tacit knowledge must be deliberately built into any system that’s going to be genuinely useful. The model needs the human to tell it what matters.
What I’ve learned about using these tools well.
I use AI dialogically – as a thinking partner, not a ghostwriter. When I ask Claude to critique its own first draft, I get a better document than when I accept the first output. The iteration is the work.
I also won’t use tools I can’t interrogate. The conversational interface of large language models is, for practitioners, a practical form of Socratic dialogue – you can ask why, push back, watch the model recalibrate. Glass box over black box, always.
The human foundation is the precondition, not the obstacle.
The organisations and practitioners who will use AI well are not the ones who deploy it most boldly. They’re the ones who understand precisely where the pattern-matching stops and the understanding must begin.
In Advancement, that boundary is everywhere relationships are. Which is everywhere that matters.
Sarah
Inferential Loop 