Our 4B Property Model Gained 6.7 Points at Q4 and None at Q8
In exp_039 we trained Qwen3-4B with GRPO against a reward that checked form: language, citations, numbers, length. The model learned what the reward paid for and did not get more correct. The post ended with a list of changes. exp_040 makes the main one: the reward checks the answer itself. Every answer now ends with one line, Answer: …, and code checks that line against a closed label: one of the question’s options, a number computed in code, or “not in the material”. A right line earns +3, a wrong or hedged line −2, a missing line −1. Language, plain text, citations, number support, length, repetition and copying stay as small terms. ...