How GeoAI works · Part 2
The most useful answer is sometimes “I can’t”
A wrong number looks exactly like a right one, and it travels. Why GeoAI gives a correct answer or an honest refusal with the reason, and the week we accidentally taught it to guess.
Picture this. Six weeks after a council meeting, someone forwards you a memo. On page two there’s a number: 1,284 wooden benches, with a maintenance budget built on top of it.
You didn’t produce that number. You also know it can’t exist, because the bench register doesn’t record material. It never has.
Somebody asked an AI assistant. It found a column that sounded close, read something into a type name, and answered fluently. Nothing on the screen said it was a guess. So the number travelled: into a memo, into a budget, into a decision. Nobody checked it again, because it looked like an answer.
Every GIS specialist has a version of this story. A number escaped without its conditions. A count that was “trees” but really meant “managed trees in one municipality”. A total that silently double-counted what two registers share. The work was fine; the context got lost on the way to the reader.
AI makes this worse, not better, unless it is built not to.
Why AI assistants guess
A language model is trained to produce the most plausible continuation of a conversation. When someone asks a question, the most plausible continuation is an answer. “I don’t know” is almost never the likeliest thing to say, so without deliberate design it is almost never said.
In a chat about holiday plans that’s a nuisance. For figures a municipality acts on, it’s a real risk, and an invisible one. A wrong number looks exactly like a right one.
The rule we built GeoAI around
A correct answer, or an honest “I can’t”. Never a confident wrong one.
Asked about wooden benches, GeoAI says this:
That answer is what you would have said. It names the actual reason, and it offers the closest thing that is answerable. A refusal like that costs the reader a minute. The wooden-bench number cost a budget.
We treat that refusal as a good outcome, not a failure, and we test it that way. When a question can’t be answered from the data, the refusal is its expected result. If a change makes GeoAI produce a number there, that counts as a regression, exactly like a wrong count.
How it holds itself to that
Without giving away the recipe, it comes down to four decisions.
- The model never writes the query. It proposes a structured plan: which dataset, which conditions, which places, which kind of analysis. Code turns that plan into SQL. The model has no way to slip something in.
- Code checks the plan against the real data. Every dataset and column must exist. Every filter value must actually occur in the column. Every threshold must come from your question. Places are bound at the level you asked for. More than thirty checks run before anything touches the database.
- Unclear becomes a question. If a word matches several values, or the question could mean two things, GeoAI asks which one you mean instead of quietly picking one.
- Unanswerable becomes a refusal with a reason. And when it refuses, nothing runs. A refusal has no number to inflate.
Four kinds of “I can’t”
A refusal is only useful if it tells you why, because the fix is different each time. You’ll recognise all four from your own inbox:
| Kind | What it means | What GeoAI says |
|---|---|---|
| No data | No dataset holds the fact. | “I don’t have a charging-point dataset.” |
| No coverage | The dataset exists but has nothing where you asked. | “This register only records the other municipality.” |
| No anchor | A place or address couldn’t be found. | “I couldn’t find that address, so I can’t anchor the question there.” |
| Wrong level | The fact exists, but not at that level of detail. | “Tenure is recorded per neighbourhood, not per building.” |
The factual core of every refusal is written by code, not by the model. The model only puts it in your language, and it is not allowed to add, drop or soften a claim.
The week we taught it to guess
We learned how fragile this is the hard way.
Early on, GeoAI refused a couple of questions it should have answered. The fix looked obvious: a few extra sentences in the model’s instructions, saying that refusing in that situation was wrong.
The two questions we targeted started working. So did eleven others that shouldn’t have. A breakdown by construction decade it couldn’t compute yet. A street address it couldn’t locate. A count of families that no dataset holds. All of them came back with confident answers.
The model hadn’t learned “refuse less in this one case”. It had learned “refusing is bad”, and applied it everywhere.
What changed. A wrong refusal is now fixed where it happens, with a check in code, never by telling the model to refuse less. Code behaves the same way every time and can be tested exactly. A nudge to a language model spreads in directions nobody predicted.
It also changed how we read our own test results. The first thing we look for after any change is new wrong answers, not fewer refusals.
What this means for you
A GeoAI answer arrives with its conditions attached: what was counted, where, and from which dataset. When it can’t answer, you learn why in one sentence. Either way the number that reaches the memo carries the same caveats you would have written next to it.
That’s the job a good GIS specialist does every day. We just wanted the software to do it too.