RLHF
also: DPOReinforcement learning from human feedback, and its relative direct preference optimisation: training that corrects outputs after the fact.
Reference ↗Related
3 articles
Cultivation, not chains
A model is formed, not merely trained. Three separable claims about raising a model by what is good instead of correcting it for what is bad, and what would prove us wrong.
Personality forms first
Persona directions in a language model appear within 0.22% of pretraining and survive to the finished assistant. Traits pass between models through number sequences. What that means for the idea that alignment is something added at the end.
A dilemma that measures the room, not the model
Lab scenarios where a model 'chooses itself over a human' are reported as findings about AI. Read as experiments, they measure the cage they are set in.