Temperature is a sampling parameter that controls the randomness of a language model's output. At a low temperature (near 0) the model almost always picks the most probable next token, producing focused, deterministic, repeatable answers that are ideal for factual tasks, code, and extraction. At a higher temperature the probability distribution is flattened, so less likely tokens get chosen more often, yielding more varied and creative but less predictable text. Temperature is often tuned alongside top-p (nucleus sampling) and top-k, which also shape which tokens can be sampled. Choosing the right temperature is a core part of getting reliable behaviour from an LLM.
Frequently Asked Questions
What does temperature do in an LLM?
Temperature controls output randomness: low values make the model deterministic and focused, higher values make it more varied and creative but less predictable.
What temperature should I use?
Use a low temperature (0 to 0.3) for factual answers, code, and extraction, and a higher one (0.7 to 1.0) for brainstorming and creative writing.