Few-shot prompting lets you achieve task‑specific performance without the compute and data costs of full fine‑tuning, by supplying a handful of exemplars that condition the model’s behavior at inference time.
1. Select representative exemplars – pick 3‑5 inputs that span the edge cases of your target distribution. Use the same format the model will see in production.
2. Structure the prompt – wrap each exemplar in a clear delimiter (e.g., ---) and prepend a concise instruction. Example:
```python
prompt = """You are a legal‑style summarizer.
---
Input: {doc1}\nSummary: {summary1}
---
Input: {doc2}\nSummary: {summary2}
---
Input: {new_doc}\nSummary:"""
```
3. Set inference flags – keep temperature ≤ 0.2 for deterministic output, max_tokens to the expected summary length, and presence_penalty = 0 to avoid hallucinations.
4. Validate with a held‑out set – compute ROUGE‑L or BLEU on 100 examples; aim for ≥ 0.75 of the score you get from a fully fine‑tuned model.
5. Iterate – if performance drops > 5 % after a context‑window change, replace the oldest exemplar with a newer, more relevant one.
Few‑Shot vs. Full Fine‑Tune (quick reference)
| Aspect | Few‑Shot Prompting | Full Fine‑Tune |
|-----------------------|-------------------|----------------|
| Compute cost | < 0.5 GPU‑hr | > 50 GPU‑hr |
| Data requirement | ≤ 10 examples | ≥ 10 k samples |
| Deployment latency | + 10 ms per request| same model size |
| Flexibility (swap tasks) | Immediate (edit prompt) | Retrain needed |
Gotcha: When the total prompt length approaches the model’s context window (e.g., 8 k tokens for Llama‑3‑8B), the last exemplar may be truncated, causing silent performance loss; always monitor prompt_tokens in the API response.