A Local Decision Model Made Our 4B Refuse 92 % of Unanswerable Questions and None of the Answerable Ones
In exp_040 we tried to train our 4B property model to say “not in the material” when the retrieved study text does not contain the rule. Training did not fix it. When the source was missing, the untrained model still answered 55 % of the time, from whatever it remembered. So we stopped asking the model to know what it does not know. Before the model answers, a separate small decision model reads the retrieved text and the question and returns one probability: can this material answer this question? If not, the system says “not in the material” and the model never answers. ...