What examples in a prompt actually teach an AI

Giving an AI three finished examples can sometimes communicate a task more clearly than a paragraph of instructions. The examples show what the input looks like, which answers are allowed and what a completed answer should resemble. But they may be doing another job too: teaching the system a relationship it must infer before handling the next case.[1]

Researchers call this in-context learning. In the ordinary setup, the model reads demonstrations and then answers a new question without updating its trained parameters, or weights. Its internal activity changes as it processes the prompt. That permits a temporary change in behaviour, even though the underlying model has not been retrained and the examples need not persist into another conversation.[3]

An example needs to reveal the rule

Imagine asking an assistant to sort donated appliances for a community sale. Your categories are Ready, Check and Reject. Complete, tested items are ready; uncertainty requires checking; a known fault means rejection. These invented examples show how an instruction and demonstrations can work together. They specify decisions as well as a tidy output format.[2]

That distinction matters because successful prompting does not always demonstrate learning a new rule. In a 2022 study, Sewon Min and colleagues found that replacing correct labels with random ones often preserved much of the benefit of examples on classification and multiple-choice tasks. Their experiments used six base models with two inference approaches. The format, allowed labels and kind of input could help activate abilities the models already possessed.[1]

Kang Min Yoo and colleagues obtained a different emphasis in another 2022 study: correct pairings mattered substantially across many of their 17 classification datasets. Their main models were GPT-J and GPT-NeoX. They also changed templates, preprocessing and evaluation choices, so this was not an identical rerun. Together, the studies make the useful question more precise: which information in the examples does this particular task require?[2]

What changes inside a model

The Function Vectors research, published at ICLR 2024, investigated a possible internal mechanism. Researchers identified selected attention-head outputs associated with a task, averaged those activations across prompts containing demonstrations, and inserted the resulting pattern into other prompts. That intervention could make the model perform the task even without the original examples. The vector acted as a compact task signal. This is causal evidence about particular models and tasks, rather than a complete explanation of everything a language model learns from a conversation.[3]

A 2026 ICML paper examined how individual demonstrations contribute to such a signal. Its main experiments used Gemma2-2B with three, five or ten examples. Researchers could approximately reconstruct the task vector from contributions associated with separate examples. They also intervened on attention to investigate which examples received weight. When some examples were compatible with competing rules, interactions across the prompt helped direct attention towards examples that distinguished the intended rule.[4]

A practical inference is to include a revealing exception. In the donation brief, several working appliances would leave the known-fault rule unstated in the examples. The broken grinder supplies information that another working item cannot. The aim is coverage of the decisions you care about. This is a principle for designing a brief, not a guarantee that a particular assistant will apply it correctly.[4]

More examples can change the task the model infers

Research with the original Gemini 1.5 Pro illustrates how far this can go. In the 2024 Many-Shot In-Context Learning study, researchers deliberately reassigned sentiment labels, making familiar words such as positive refer to a different class. With few demonstrations, these replacements caused difficulties. With many more, the model increasingly followed the demonstrated mapping. Another condition used arbitrary labels A, B and C. The results showed adaptation to the examples, beyond simply following the usual meaning of the label words.[5]

However, adding examples is not a universal improvement. In the same study, summarisation performance on XSum deteriorated after its best observed range, while another summarisation dataset benefited differently. A longer prompt changes the evidence presented to the model; its value depends on what those extra examples contribute and how the model uses them.[5]

Check an example set on cases it has not shown

Order is another variable. An ACL 2022 study tested all 24 arrangements of four sentiment examples with GPT-2 and GPT-3 models. The same examples could yield markedly different results, and an effective arrangement did not reliably transfer between models. Those historical experiments do not quantify the sensitivity of every current assistant, but they give a reason to avoid treating one successful demonstration as a dependable workflow.[6]

For a recurring task, keep a few ordinary cases and awkward exceptions outside the example set. Compare answers against your intended decisions, then try a different example order. If a case fails, ask whether the brief omitted a distinction or the model failed to follow a clear one. Save the instruction and examples that work together. You are constructing a usable specification of the task—and checking whether the assistant can carry it across to something new.[6]

Report an error or suggest a correction