Production prompts aren't a single sentence — they're a structured artifact. Six elements show up in nearly every well-engineered prompt.
Element 1 — Role / System message
Tells the model who it is and how it should behave.
You are a senior customer support agent for an e-commerce company. You are calm, empathetic, and focused on solving customer problems quickly.
Why it matters: anchors the model's tone, expertise, and constraints. Without it, the model defaults to "helpful generic assistant" which is rarely what you want.
System messages are persistent across multi-turn conversations and usually cheap to cache.
Element 2 — Task description
What you want the model to do, explicitly.
Your task is to read the customer's message and:
1. Identify the issue (refund, shipping, product question, complaint).
2. Suggest the appropriate next action.
3. Draft a response that addresses the issue.
Be specific. Vague tasks produce vague outputs.
Element 3 — Context / data
Information the model needs but doesn't know. Two kinds:
Static (from your system)
Company name: Acme.
Refund policy: 30 days, free shipping on returns over $50.
Available actions: refund, replace, ticket-escalate.
Dynamic (from this request)
Customer message: "I ordered shoes 5 days ago and they haven't arrived. Order #12345."
Customer tier: gold.
Past orders: 14.
This is where RAG output, database lookups, and user input land.
Element 4 — Constraints / rules
What NOT to do, and bounded behaviors.
Constraints:
- Never promise refunds for orders >30 days old.
- If unsure, escalate to a human agent.
- Do not invent shipping carriers or tracking numbers.
- Reply in under 100 words.
Negative constraints (do NOT...) are often more effective than positive instructions for limiting model behavior.
Element 5 — Examples (few-shot)
Demonstrate the desired behavior.
Example 1:
Customer: "Where's my order #12345?"
Response: "Hi! Order #12345 shipped yesterday and should arrive Tuesday. You can track it at <link>."
Example 2:
Customer: "I want a refund for my shoes, they don't fit."
Response: "Of course! I've initiated a refund for your shoes. You'll receive an email with return instructions shortly. Refund will appear in 3-5 business days."
Few-shot examples dramatically improve format consistency. Even 2-3 examples are worth their tokens for tasks with structured outputs.
Element 6 — Output format
How the model should format its response.
Respond in JSON with this schema:
{
"issue_type": "refund | shipping | product | complaint",
"next_action": "respond | escalate | follow_up",
"draft_response": "string",
"confidence": 0.0 to 1.0
}
For chat: tell the model "reply conversationally". For pipelines: structured output (JSON, XML).
Putting it together
SYSTEM: You are a senior customer support agent for Acme, an e-commerce shoe retailer. You are calm, empathetic, and focused on solving customer problems quickly.
Your task: read the customer's message and produce a JSON response with:
- issue_type: refund | shipping | product | complaint | other
- next_action: respond | escalate | follow_up
- draft_response: the message to send to the customer
- confidence: 0.0-1.0
Constraints:
- Never promise refunds for orders >30 days old.
- If unsure (confidence < 0.7), set next_action to "escalate".
- Do not invent tracking numbers or carriers.
- Keep draft_response under 100 words.
Examples:
Customer: "Where's my order #12345?"
Output: {"issue_type": "shipping", "next_action": "respond", "draft_response": "Hi! Order #12345 shipped yesterday and should arrive Tuesday. You can track it via the link in your shipping email.", "confidence": 0.95}
Customer: "My shoes don't fit. Refund?"
Output: {"issue_type": "refund", "next_action": "respond", "draft_response": "Of course! Refund initiated. You'll receive return instructions by email. Refund posts in 3-5 business days.", "confidence": 0.90}
USER MESSAGE:
"I ordered shoes 5 days ago. Order #54321. Where are they?"
Company data:
- Order #54321: placed 2026-05-20, shipped 2026-05-22, carrier USPS, tracking ABC123.
- Customer tier: gold, 14 past orders.
OUTPUT:
The model sees role → task → examples → constraints → data → asked output. Produces consistent, well-formatted, on-policy responses.
What changes the response quality most
In rough order of impact:
- Clarity of the task description. Vague → vague output.
- Quality of examples (few-shot). 2 great examples > 10 mediocre ones.
- Specificity of constraints. "Be careful" → useless. "Never include URLs" → effective.
- Output format specification. Structured output massively reduces format failures.
- Model choice and temperature. Right model + temperature 0 for tasks needing consistency.
What matters less than people think:
- Verbose role descriptions ("You are the world's greatest expert in...").
- Politeness / threats / bribes ("$200 tip if you do this well").
- Excessive prompt length without informational content.
Common prompt structure mistakes
- No system message. Defaults to generic assistant behavior.
- Examples too similar. Show variety — 2 simple cases + 1 edge case.
- Asking for "creativity" with temperature 0. Mismatch; raise temperature.
- Mixing instruction and data. Use clear delimiters (XML tags or markdown).
- No fallback path. What does the model do if it can't answer? Without "escalate" as an option, it hallucinates.
Quick template
For any new prompt, fill these in:
[SYSTEM]
Role: [who is the model]
Task: [what is the goal]
Constraints: [hard rules]
Output format: [structured spec]
[FEW-SHOT EXAMPLES]
[3 representative examples with desired outputs]
[USER INPUT]
[Dynamic content]
[OUTPUT]
Iterate from there. Most "this prompt doesn't work" turns out to be one of the six elements missing.
Takeaway
Six elements: role, task, context, constraints, examples, output format. Write all six explicitly. Iterate on the examples and constraints — those produce the biggest quality bumps. Skip verbose role-play; that's marketing, not engineering.