Build your own decision model
- "System one" decision models are models that infer and respond with calibrated probabilities or every allowed answer.
- Consider your everyday language model, to get typed output from it (JSON), you may use Structured Output to constrain the output to guaranteed valid JSON.
- While model prefills the input in one pass, it still has to go perform a pass for every token in order to generate a valid response.
Unverified
- "System one" decision models are models that infer and respond with calibrated probabilities or every allowed answer.
- Consider your everyday language model, to get typed output from it (JSON), you may use Structured Output to constrain the output to guaranteed valid JSON.
- While model prefills the input in one pass, it still has to go perform a pass for every token in order to generate a valid response.
Sources: Nishtahir