I’ve been building a typing assistant for a Socionics matching app, and the interesting constraint was this: the model has to work through a formal function-stack framework rather than pattern-match to stereotypes, and it has to be honest when it’s genuinely uncertain instead of confidently mistyping someone.
A few design decisions that might be useful to others doing structured-reasoning chat:
- Adaptive over fixed. It replaced a static questionnaire. The whole point is that follow-ups branch on what the user actually says, so when answers point somewhere unexpected the conversation can chase that rather than ploughing through a script.
- Confidence as a first-class output. Rather than returning a single label, it produces a lean across likely candidates. Below a confidence threshold it explicitly offers a choice rather than asserting — which matters when the ground truth is genuinely hard and self-report is unreliable.
- Reasoning against a source of truth. Type and relation data come from canonical data files, not the model’s own recall, to stop it drifting into confident nonsense.
The honest-uncertainty piece was the hardest part — getting a model to say “I’m leaning X but genuinely not sure between X and Y, here’s why” instead of picking one and committing. Happy to talk through how that was prompted.
Curious how others handle the “don’t hallucinate confidence” problem in domains where the correct answer is fuzzy.
submitted by /u/SpencerStern
[link] [comments]