Truth for believable AI: the third Perceived AGI paper
Paper three of the Perceived AGI series is on arXiv. It asks whether an AI assistant can be made to say when it is not sure, where a claim comes from, and when it has changed its mind.
The third article in the Perceived AGI series is now on arXiv: Truth for Believable AI. It follows Perceived AGI, the framework paper, and Entropy in Conversational AI.
AI assistants answer everything in the same confident tone. A fact they know and a guess sound exactly alike. People don’t talk like that. We say “I’m not sure”, we say where we heard something, and we change our mind when someone shows us we were wrong.
The paper tries to turn those three habits into something you can build. It puts a thin layer on top of an unchanged model. For every claim, the layer keeps a confidence score and a note on where the claim came from: what the model already knew, something it looked up, something it worked out, or something you told it. From those two it decides whether to state the answer, hedge, or say it doesn’t know. It also remembers corrections from one conversation to the next, says when it has changed its mind, and does not simply accept every correction it is given.
On the small model I tested, some of this worked. The layer cut confidently wrong answers to about a fifth. The cost was that it answered fewer questions, and the answers it did give were just as accurate. It kept a record you can check of every time it changed its mind, and it accepted true corrections more often than false ones, though many false ones still got through. Other parts failed. Using the model’s own word probabilities to judge how sure it is turned out worse than a coin flip at telling right answers from wrong ones. A check I added afterwards found that asking the same question several times and seeing whether the answers agree works better. The paper reports the failures along with the rest, and says the test needs far more than the 60 facts I used before it goes to people.
What I want is an assistant that is honest about how sure it is: when to doubt, what it relies on, and when to change its position.
The series takes believability one dimension at a time: time, entropy, truth, and love. It ends with a study with people, and that is where trust gets tested. The code and benchmark are public, so anyone can rerun this on a bigger model.
